A bunch of code is wrapped in #if WITH_LIBVIRTD in order to
enable the virStateDriver to be disabled when libvirtd is not
built. Disabling this code doesn't have any real functional
benefit beyond removing 1 pointer from the virConnectPtr struct,
while having a cost of many more conditionals.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since our big split of libvirt.c there are only a few functions
living there. The majority was moved to corresponding subfile,
e.g. domain functions were moved to libvirt-domain.c. However,
the patches for virDomainGetFSInfo() and virDomainFSInfoFree()
introduction were posted prior the big split and merged after.
This resulted in two domain functions landing in wrong file.
Move them to the correct one.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

virDomainGetFSInfo returns a list of filesystems information mounted in the
guest, which contains mountpoints, device names, filesystem types, and
device aliases named by libvirt. This will be useful, for example, to
specify mountpoints to fsfreeze when taking snapshot of a part of disks.
Signed-off-by: NTomoki Sekiyama <tomoki.sekiyama@hds.com>

C guarantees that static variables are zero-initialized.  Some older
compilers (and also gcc -fno-zero-initialized-in-bss) create larger
binaries if you explicitly zero-initialize a static variable.

* src/libvirt.c: Fix initialization.
* src/util/viralloc.c: Likewise.
* src/util/virdbus.c: Likewise.
* src/util/virevent.c: Likewise.
* src/util/virfile.c (safezero): Likewise.
* src/util/virlog.c: Likewise.
* src/util/virnetlink.c: Likewise.
* src/util/virthread.h (VIR_ONCE_GLOBAL_INIT): Likewise.
* src/util/virprocess.c (virProcessGetStartTime): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Introduce a src/libvirt-host.c file to hold all the
methods related to the virConnect type.

Introduce a src/libvirt-domain.c file to hold all the
methods related to the virDomain type.

The virTypedParameterValidateSet method will need to be used
from several libvirt-*.c files so must be non-static
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce a src/libvirt-storage.c file to hold all the
methods related to the virStorage{Pool,Vol} types.

Introduce a src/libvirt-stream.c file to hold all the
methods related to the virStream type.

Introduce a src/libvirt-secret.c file to hold all the
methods related to the virSecret type.

Introduce a src/libvirt-nodedev.c file to hold all the
methods related to the virNodeDevice type.

Introduce a src/libvirt-nwfilter.c file to hold all the
methods related to the virNWFilter type.

Introduce a src/libvirt-interface.c file to hold all the
methods related to the virInterface type.

Introduce a src/libvirt-network.c file to hold all the
methods related to the virNetwork type.

Introduce a src/libvirt-domain-snapshot.c file to hold all the
methods related to the virDomainSnapshot type.

To prepare for introducing a single global driver, rename the
virDriver struct to virHypervisorDriver and the registration
API to virRegisterHypervisorDriver()

Management software wants to be able to allocate disk space on demand.
To support this they need keep track of the space occupation of the
block device.  This information is reported by qemu as part of block
stats.

This patch extend the block information in the bulk stats with the
allocation information.

To keep the same behaviour a helper is extracted from
qemuMonitorJSONGetBlockExtent in order to get per-device allocation
information.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Due to a missing check the API can be successfully called even if
the connection is ReadOnly. Fortunately, the API hasn't been
released yet, so there's no need for a CVE.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

A long time ago in a galaxy far, far away it has been decided
that libvirt will manage not only domains but host as well. And
with my latest work on qemu driver supporting huge pages, we miss
the cherry on top: an API to allocate huge pages on the run.
Currently users are forced to log into the host and adjust the
huge pages pool themselves.  However, with this API the problem
is gone - they can both size up and size down the pool.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NJincheng Miao <jmiao@redhat.com>

Clean up all _virDomainBlockStats.
Signed-off-by: NJames <james.wangyufei@huawei.com>
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Clean up all _virDomainInterfaceStats.
Signed-off-by: NWang Yufei <james.wangyufei@huawei.com>
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Add a note to make the users aware that some stats groups or fields may
be missing in certain cases.

State that full stats for the stats groups are available in the
virConnectGetAllDomainStats documentation section rather than
duplicating the docs.

This patch implements the VIR_DOMAIN_STATS_BLOCK group of statistics.

To do so, a helper function to get the block stats of all the disks of
a domain is added.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

This patch implements the VIR_DOMAIN_STATS_INTERFACE group of
statistics.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

This patch implements the VIR_DOMAIN_STATS_VCPU group of statistics. To
do so, this patch also extracts a helper to gather the vCPU information.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

This patch implements the VIR_DOMAIN_STATS_BALLOON group of statistics.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>

This patch implements the VIR_DOMAIN_STATS_CPU_TOTAL group of
statistics.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

We stupidly modeled block job bandwidth after migration
bandwidth, which in turn was an 'unsigned long' and therefore
subject to 32-bit vs. 64-bit interpretations.  To work around
the fact that 10-gigabit interfaces are possible but don't fit
within 32 bits, the original interface took the number scaled
as MiB/sec.  But this scaling is rather coarse, and it might
be nice to tune bandwidth finer than in megabyte chunks.

Several of the block job calls that can set speed are fed
through a common interface, so it was easier to adjust them all
at once.  Note that there is intentionally no flag for the new
virDomainBlockCopy; there, since the API already uses a 64-bit
type always, instead of a possible 32-bit type, and is brand
new, it was easier to just avoid scaling issues.  As with the
previous patch that adjusted the query side (commit db33cc24),
omitting the new flag preserves old behavior, and the
documentation now mentions limits of what happens when a 32-bit
machine is on either client or server side.

* include/libvirt/libvirt.h.in (virDomainBlockJobSetSpeedFlags)
(virDomainBlockPullFlags)
(VIR_DOMAIN_BLOCK_REBASE_BANDWIDTH_BYTES)
(VIR_DOMAIN_BLOCK_COMMIT_BANDWIDTH_BYTES): New enums.
* src/libvirt.c (virDomainBlockJobSetSpeed, virDomainBlockPull)
(virDomainBlockRebase, virDomainBlockCommit): Document them.
* src/qemu/qemu_driver.c (qemuDomainBlockJobSetSpeed)
(qemuDomainBlockPull, qemuDomainBlockRebase)
(qemuDomainBlockCommit, qemuDomainBlockJobImpl): Support new flag.
Signed-off-by: NEric Blake <eblake@redhat.com>

Total time of a migration and total downtime transfered from a source to
a destination host do not count with the transfer time to the
destination host and with the time elapsed before guest CPUs are
resumed. Thus, source libvirtd remembers when migration started and when
guest CPUs were paused. Both timestamps are transferred to destination
libvirtd which uses them to compute total migration time and total
downtime. Obviously, this requires the time to be synchronized between
the two hosts. The reported times are useless otherwise but they would
be equally useless if we didn't do this recomputation so don't lose
anything by doing it.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

virDomainGetJobStats gains new VIR_DOMAIN_JOB_STATS_COMPLETED flag that
can be used to fetch statistics of a completed job rather than a
currently running job.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

To date, anyone performing a block copy and pivot ends up with
the destination being treated as <disk type='file'>.  While this
works for data access for a block device, it has at least one
noticeable shortcoming: virDomainGetBlockInfo() reports allocation
differently for block devices visited as files (the size of the
device) than for block devices visited as <disk type='block'>
(the maximum sector used, as reported by qemu); and this difference
is significant when trying to manage qcow2 format on block devices
that can be grown as needed.

Of course, the more powerful virDomainBlockCopy() API can already
express the ability to set the <disk> type.  But a new API can't
be backported, while a new flag to an existing API can; and it is
also rather inconvenient to have to resort to the full power of
generating XML when just adding a flag to the older call will do
the trick.  So this patch enhances blockcopy to let the user flag
when the resulting XML after the copy must list the device as
type='block'.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BLOCK_REBASE_COPY_DEV):
New flag.
* src/libvirt.c (virDomainBlockRebase): Document it.
* tools/virsh-domain.c (opts_block_copy, blockJobImpl): Add
--blockdev option.
* tools/virsh.pod (blockcopy): Document it.
* src/qemu/qemu_driver.c (qemuDomainBlockRebase): Allow new flag.
(qemuDomainBlockCopy): Remember the flag, and make sure it is only
used on actual block devices.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

While reviewing the new virDomainBlockCopy API, Peter Krempa
pointed out that our existing design of using MiB/s for block
job bandwidth is rather coarse, especially since qemu tracks
it in bytes/s; so virDomainBlockCopy only accepts bytes/s.
But once the new API is implemented for qemu, we will be in
the situation where it is possible to set a value that cannot
be accurately reflected back to the user, because the existing
virDomainGetBlockJobInfo defaults to the coarser units.

Fortunately, we have an escape hatch; and one that has already
served us well in the past: we can use the flags argument to
specify which scale to use (see virDomainBlockResize for prior
art).  This patch fixes the query side of the API; made easier
by previous patches that split the query side out from the
modification code.  Later patches will address the virsh
interface, as well retrofitting all other blockjob APIs to
also accept a flag for toggling bandwidth units.

* include/libvirt/libvirt.h.in (_virDomainBlockJobInfo)
(VIR_DOMAIN_BLOCK_COPY_BANDWIDTH): Document sizing issues.
(virDomainBlockJobInfoFlags): New enum.
* src/libvirt.c (virDomainGetBlockJobInfo): Document new flag.
* src/qemu/qemu_monitor.h (qemuMonitorBlockJobInfo): Add parameter.
* src/qemu/qemu_monitor.c (qemuMonitorBlockJobInfo): Likewise.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONBlockJobInfo):
Likewise.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONBlockJobInfo)
(qemuMonitorJSONGetBlockJobInfoOne): Likewise. Don't scale here.
* src/qemu/qemu_migration.c (qemuMigrationDriveMirror): Update
callers.
* src/qemu/qemu_driver.c (qemuDomainBlockPivot)
(qemuDomainBlockJobImpl): Likewise.
(qemuDomainGetBlockJobInfo): Likewise, and support new flag.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add domain list filtering functions and a flag to enforce checking
whether the remote daemon supports the requested stats groups.

Let's fix this before we bake in a painful API.  Since we know
that we have exactly one non-negative fd on success, we might
as well return the fd directly instead of forcing the user to
pass in a pointer.  Furthermore, I found some memory and fd
leaks while reviewing the code - the idea is that on success,
libvirtd will have handed two fds in two different directions:
one to qemu, and one to the RPC client.

* include/libvirt/libvirt.h.in (virDomainOpenGraphicsFD): Drop
unneeded parameter.
* src/driver.h (virDrvDomainOpenGraphicsFD): Likewise.
* src/libvirt.c (virDomainOpenGraphicsFD): Adjust interface to
return fd directly.
* daemon/remote.c (remoteDispatchDomainOpenGraphicsFd): Adjust
semantics.
* src/qemu/qemu_driver.c (qemuDomainOpenGraphicsFD): Likewise,
and plug fd leak.
* src/remote/remote_driver.c (remoteDomainOpenGraphicsFD):
Likewise, and plug memory and fd leak.
Signed-off-by: NEric Blake <eblake@redhat.com>

This commit (finally) adds the virDomainBlockCopy API, with the
intent that it will provide more power to the existing 'virsh
blockcopy' command.

'virsh blockcopy' was first added in Apr 2012 (v0.9.12), which
corresponds to the upstream qemu 1.2 timeframe.  It was done as
a hack on top of the existing virDomainBlockRebase() API call,
for two reasons: 1) it was targetting a feature that landed first
in downstream RHEL qemu, but had not stabilized in upstream qemu
at the time (and indeed, 'drive-mirror' only landed upstream in
qemu 1.3 with slight differences to the first RHEL attempt,
and later gained further parameters like granularity and buf-size
that are also worth exposing), and 2) extending an existing API
allowed it to be backported without worrying about bumping .so
versions.  A virDomainBlockCopy() API was proposed at that time
[1], but we decided not to accept it into libvirt until after
upstream qemu stabilized, and it ended up getting scrapped.
Whether or not RHEL should have attempted adding a new feature
without getting it upstream first is a debate that can be held
another day; but enough time has now elapsed that we are ready to
do the interface cleanly.

[1] https://www.redhat.com/archives/libvir-list/2012-April/msg00768.html

Delaying the creation of a clean API until now has also had a
benefit: we've only recently learned of a few shortcomings in the
original design: 1) it is unable to target a network destination
(such as a gluster volume) because it hard-coded the assumption
that the destination is a local file name.  Because of all the
refactoring we've done to add virStorageSourcePtr, we are in a
better position to declare an API that parses XML describing a
host storage source as the copy destination, which was not
possible had we implemented virDomainBlockCopy as it had been
originally envisioned (although a network target will have to wait
until a later libvirt release compared to the API addition to
actually be implemented).  2) the design of using MiB/sec as the
bandwidth throttle is rather coarse; qemu is actually tuned to
bytes/second, and libvirt is preventing access to that level of
detail.  A later patch will add flags to existing block job API
that can request bytes/second instead of back-compat MiB/s, but as
this is a new API, we can get it right to begin with.

At least I had the foresight to create 'virsh blockcopy' as a
separate command at the UI level (commit 1f06c007) rather than
leaking the underlying API overload of virDomainBlockRebase onto
shell users.

A further note on the bandwidth option: virTypedParameters
intentionally lacks unsigned long (since variable-width
interaction between mixed 32- vs. 64-bit client/server setups is
nasty), but we have to deal with the fact that we are interacting
with existing older code that mistakenly chose unsigned long
bandwidth at a point before we decided to prohibit it in all new
API.  The typed parameter is therefore unsigned long long, but
the implementation (in a later patch) will have to do overflow
detection on 32-bit platforms, as well as capping the value to
match the LLONG_MAX>>20 cap of the existing MiB/s interfaces.

* include/libvirt/libvirt.h.in (virDomainBlockCopy): New API.
(virDomainBlockJobType, virConnectDomainEventBlockJobStatus):
Update related documentation.
* src/libvirt.c (virDomainBlockCopy): Implement it.
* src/libvirt_public.syms (LIBVIRT_1.2.8): Export it.
* src/driver.h (_virDriver): New driver callback.
Signed-off-by: NEric Blake <eblake@redhat.com>

The motivation for this API is that management layers that use libvirt
usually poll for statistics using various split up APIs we currently
provide. To get all the necessary stuff, the app needs to issue a lot of
calls and aggregate the results.

The APIs I'm introducing here:
1) Returns data in a format that we can expand in the future and is
(pseudo) hierarchical. The data is returned as typed parameters where
the fields are constructed as dot-separated strings containing names and
other stuff in a list of typed params.

2) Stats for multiple (all) domains can be queried at once and are
returned in one call. This will decrease the overhead necessary to issue
multiple calls per domain multiplied by the count of domains.

3) Selectable (bit mask) fields in the returned format. This will allow
to retrieve only specific stats according to the app's need.

The stats groups will be enabled using a bit field @stats passed as the
function argument. A few sample stats groups that this API will support:

VIR_DOMAIN_STATS_STATE
VIR_DOMAIN_STATS_CPU
VIR_DOMAIN_STATS_BLOCK
VIR_DOMAIN_STATS_INTERFACE

(Note that this is only an example, the initial implementation supports
 only VIR_DOMAIN_STATS_STATE while others will be added later.)

the returned typed params will use the following scheme

state.state = VIR_DOMAIN_RUNNING
state.reason = VIR_DOMAIN_RUNNING_BOOTED (the actual values according to
                                          the enum)
cpu.count = 8
cpu.0.state = running
cpu.0.time = 1234

Define the public API implementation and declare internal
driver prototype.