Add a @detach parameter to the API in order allow running the QEMU
code as a thread.

Reviewed-by: Jiri Denemark <jdenemar redhat com>

Add the query-dump API's in order to allow the dump-guest-memory
to be used to monitor progress. This will use the dump stats
extraction helper to fill a return buffer.
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

The event will be fired when the domain memory only dump completes.

Fill in a return buffer to store/pass along the dump statistics that
will be eventually shared by a query-dump command. Also pass along
the status of the filling and any possible error received.
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Define the qemuMonitorDumpStats as a new job JobStatsType to handle
being able to get memory dump statistics. For now do nothing with
the new TYPE_MEMDUMP.
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

This patch pass event error up to the place where we can
use it. Error is passed only for sync blockjob event mode
as we can't use the error in async mode. In async mode we
just pass the event details to the client thru event API
but current blockjob event API can not carry extra parameter.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

We handle incremental storage migration in a different way. The support
for this new (as of QEMU 2.10) parameter is only needed for full
coverage of migration parameters used by QEMU.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

We already support several ways of setting migration bandwidth and this
is not adding another one. With this patch we are able to read and write
this parameter using query-migrate-parameters and migrate-set-parameters
in one call with all other parameters.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

The parameters used "migrate" prefix which is pretty redundant and
qemuMonitorMigrationParams structure is our internal representation of
QEMU migration parameters and it is supposed to use names which match
QEMU names.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

This new capability enables a pause before device state serialization so
that we can finish all block jobs without racing with the end of the
migration. The pause is indicated by "pre-switchover" state. Once we're
done QEMU enters "device" migration state.

This patch just defines the new capability and QEMU migration states and
their mapping to our job states.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The only remaining user of qemuMonitorGetMigrationCapability is our test
suite. Let's replace qemuMonitorGetMigrationCapability with
qemuMonitorGetMigrationCapabilities there and drop the unused function.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

So we have a syntax-check rule to catch all tab indents but it naturally
can't catch tab spacing, i.e. as a delimiter. This patch is a result of
running 'vim -en +retab +wq'
(using tabstop=8 softtabstop=4 shiftwidth=4 expandtab) on each file from
a list generated by the following:
find . -regextype gnu-awk \
         -regex ".*\.(rng|syms|html|s?[ch]|py|pl|php(\.code)?)(\.in)?" \
         | xargs git grep -lP "\t"
Signed-off-by: NErik Skultety <eskultet@redhat.com>

When migration fails, QEMU may provide a description of the error in
the reply to query-migrate QMP command. We can fetch this error and use
it instead of the generic "unexpectedly failed" message.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

query-cpu-definitions QMP command returns a list of unavailable features
which prevent CPU models from being usable on the current host. So far
we only checked whether the list was empty to mark CPU models as
(un)usable. This patch parses all unavailable features for each CPU
model and stores them in virDomainCapsCPUModel as a list of usability
blockers.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

The command "info migrate" of qemu outputs the dirty-pages-rate during
migration, but page size is different in different architectures. So
page size should be output to calculate dirty pages in bytes.

Page size is already implemented with commit
030ce1f8612215fcbe9d353dfeaeb2937f8e3f94 in qemu.
Now Implement the counter-part in libvirt.
Signed-off-by: NChao Fan <fanc.fnst@cn.fujitsu.com>
Signed-off-by: NLi Zhijian <lizhijian@cn.fujitsu.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1447169

Since domain can have at most one watchdog it simplifies things a
bit. However, since we must be able to set the watchdog action as
well, new monitor command needs to be used.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

Drop all the monitor code necessary to do the downstream block jobs.
Reviewed-by: NEric Blake <eblake@redhat.com>

Add code to support querying maximum allowable downtime during live migration.

Allow getting the raw data from query-blockstats, so that we can use it
to detect the backing chain later on.
Reviewed-by: NEric Blake <eblake@redhat.com>

vcpu properties gathered from query-hotpluggable cpus need to be passed
back to qemu. As qemu did not use the node-id property until now and
libvirt forgot to pass it back properly (it was parsed but not passed
around) we did not honor this.

This patch adds node-id to the structures where it was missing and
passes it around as necessary.

The test data was generated with a VM with following config:
    <numa>
      <cell id='0' cpus='0,2,4,6' memory='512000' unit='KiB'/>
      <cell id='1' cpus='1,3,5,7' memory='512000' unit='KiB'/>
    </numa>

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1452053

QEMU will likely report the details of it shutting down, particularly
whether the shutdown was initiated by the guest or host.  We should
forward that information along, at least for shutdown events.  Reset
has that as well, however that is not a lifecycle event and would add
extra constants that might not be used.  It can be added later on.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1384007Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Implement qemuMonitorRegister() as there is already a
qemuMonitorUnregister() function. This way it may be easier to
understand the code paths.
Signed-off-by: NMarc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: NBjoern Walk <bwalk@linux.vnet.ibm.com>

There were multiple race conditions that could lead to segmentation
faults. The first precondition for this is qemuProcessLaunch must fail
sometime shortly after starting the new QEMU process. The second
precondition for the segmentation faults is that the new QEMU process
dies - or to be more precise the QEMU monitor has to be closed
irregularly. If both happens during qemuProcessStart (starting a
domain) there are race windows between the thread with the event
loop (T1) and the thread that is starting the domain (T2).

First segmentation fault scenario:
If qemuProcessLaunch fails during qemuProcessStart the code branches
to the 'stop' path where 'qemuMonitorSetDomainLog(priv->mon, NULL,
NULL, NULL)' will set the log function of the monitor to NULL (done in
T2). In the meantime the event loop of T1 will wake up with an EOF
event for the QEMU monitor because the QEMU process has died. The
crash occurs if T1 has checked 'mon->logFunc != NULL' in qemuMonitorIO
just before the logFunc was set to NULL by T2. If this situation
occurs T1 will try to call mon->logFunc which leads to the
segmentation fault.

Solution:
Require the monitor lock for setting the log function.

Backtrace:
0  0x0000000000000000 in ?? ()
1  0x000003ffe9e45316 in qemuMonitorIO (watch=<optimized out>,
fd=<optimized out>, events=<optimized out>, opaque=0x3ffe08aa860) at
../../src/qemu/qemu_monitor.c:727
2  0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=<optimized
out>, fds=0x2aa000fd980) at ../../src/util/vireventpoll.c:508
3  0x000003fffda2e398 in virEventPollRunOnce () at
../../src/util/vireventpoll.c:657
4  0x000003fffda2ca10 in virEventRunDefaultImpl () at
../../src/util/virevent.c:314
5  0x000003fffdba9366 in virNetDaemonRun (dmn=0x2aa000cc550) at
../../src/rpc/virnetdaemon.c:818
6  0x000002aa00024668 in main (argc=<optimized out>, argv=<optimized
out>) at ../../daemon/libvirtd.c:1541

Second segmentation fault scenario:
If qemuProcessLaunch fails it will unref the log context and with
invoking qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL)
qemuDomainLogContextFree() will be invoked. qemuDomainLogContextFree()
invokes virNetClientClose() to close the client and cleans everything
up (including unref of _virLogManager.client) when virNetClientClose()
returns. When T1 is now trying to report 'qemu unexpectedly closed the
monitor' libvirtd will crash because the client has already been
freed.

Solution:
As the critical section in qemuMonitorIO is protected with the monitor
lock we can use the same solution as proposed for the first
segmentation fault.

Backtrace:
0  virClassIsDerivedFrom (klass=0x3100979797979797,
parent=0x2aa000d92f0) at ../../src/util/virobject.c:169
1  0x000003fffda659e6 in virObjectIsClass (anyobj=<optimized out>,
klass=<optimized out>) at ../../src/util/virobject.c:365
2  0x000003fffda65a24 in virObjectLock (anyobj=0x3ffe08c1db0) at
../../src/util/virobject.c:317
3  0x000003fffdba4688 in
virNetClientIOEventLoop (client=client@entry=0x3ffe08c1db0,
thiscall=thiscall@entry=0x2aa000fbfa0) at
../../src/rpc/virnetclient.c:1668
4  0x000003fffdba4b4c in
virNetClientIO (client=client@entry=0x3ffe08c1db0,
thiscall=0x2aa000fbfa0) at ../../src/rpc/virnetclient.c:1944
5  0x000003fffdba4d42 in
virNetClientSendInternal (client=client@entry=0x3ffe08c1db0,
msg=msg@entry=0x2aa000cc710, expectReply=expectReply@entry=true,
nonBlock=nonBlock@entry=false) at ../../src/rpc/virnetclient.c:2116
6  0x000003fffdba6268 in
virNetClientSendWithReply (client=0x3ffe08c1db0, msg=0x2aa000cc710) at
../../src/rpc/virnetclient.c:2144
7  0x000003fffdba6e8e in virNetClientProgramCall (prog=0x3ffe08c1120,
client=<optimized out>, serial=<optimized out>, proc=<optimized out>,
noutfds=<optimized out>, outfds=0x0, ninfds=0x0, infds=0x0,
args_filter=0x3fffdb64440
<xdr_virLogManagerProtocolDomainReadLogFileArgs>, args=0x3ffffffe010,
ret_filter=0x3fffdb644c0
<xdr_virLogManagerProtocolDomainReadLogFileRet>, ret=0x3ffffffe008) at
../../src/rpc/virnetclientprogram.c:329
8  0x000003fffdb64042 in
virLogManagerDomainReadLogFile (mgr=<optimized out>, path=<optimized
out>, inode=<optimized out>, offset=<optimized out>, maxlen=<optimized
out>, flags=0) at ../../src/logging/log_manager.c:272
9  0x000003ffe9e0315c in qemuDomainLogContextRead (ctxt=0x3ffe08c2980,
msg=0x3ffffffe1c0) at ../../src/qemu/qemu_domain.c:4422
10 0x000003ffe9e280a8 in qemuProcessReadLog (logCtxt=<optimized out>,
msg=msg@entry=0x3ffffffe288) at ../../src/qemu/qemu_process.c:1800
11 0x000003ffe9e28206 in qemuProcessReportLogError (logCtxt=<optimized
out>, msgprefix=0x3ffe9ec276a "qemu unexpectedly closed the monitor")
at ../../src/qemu/qemu_process.c:1836
12 0x000003ffe9e28306 in
qemuProcessMonitorReportLogError (mon=mon@entry=0x3ffe085cf10,
msg=<optimized out>, opaque=<optimized out>) at
../../src/qemu/qemu_process.c:1856
13 0x000003ffe9e452b6 in qemuMonitorIO (watch=<optimized out>,
fd=<optimized out>, events=<optimized out>, opaque=0x3ffe085cf10) at
../../src/qemu/qemu_monitor.c:726
14 0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=<optimized
out>, fds=0x2aa000fd980) at ../../src/util/vireventpoll.c:508
15 0x000003fffda2e398 in virEventPollRunOnce () at
../../src/util/vireventpoll.c:657
16 0x000003fffda2ca10 in virEventRunDefaultImpl () at
../../src/util/virevent.c:314
17 0x000003fffdba9366 in virNetDaemonRun (dmn=0x2aa000cc550) at
../../src/rpc/virnetdaemon.c:818
18 0x000002aa00024668 in main (argc=<optimized out>, argv=<optimized
out>) at ../../daemon/libvirtd.c:1541

Other code parts where the same problem was possible to occur are
fixed as well (qemuMigrationFinish, qemuProcessStart, and
qemuDomainSaveImageStartVM).
Signed-off-by: NMarc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reported-by: NSascha Silbe <silbe@linux.vnet.ibm.com>

If calling query-cpu-model-expansion on the 'host'/'max' CPU model with
'migratable' property set to false succeeds, we know QEMU is able to
tell us which features would disable migration. Thus we can mark all
enabled features as migratable.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

QEMU is able to tell us whether a CPU feature would block migration or
not. This patch adds support for storing such features in
qemuMonitorCPUModelInfo.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The hyperv panic notifier reports additional data in form of 5 registers
that are reported in the crash event from qemu. Log them into the VM log
file and report them as a warning so that admins can see the cause of
crash of their windows VMs.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1426176

For certain kinds of panic notifiers (notably hyper-v) qemu is able to
report some data regarding the crash passed from the guest.

Make the data accessible to the callback in qemu so that it can be
processed further.

Add monitor tooling for calling query-named-block-nodes. The monitor
returns the data as the raw JSON array that is returned from the
monitor.

Unfortunately the logic to extract the node names for a complete backing
chain will be so complex that I won't be able to extract any meaningful
subset of the data in the monitor code.

Add code to call the appropriate monitor command and code to lookup the
given disk backing chain member.

The event is fired when a given block backend node (identified by the
node name) experiences a write beyond the bound set via
block-set-write-threshold QMP command. This wires up the monitor code to
extract the data and allow us receiving the events and the capability.

Add the fields to support setting tls-creds and tls-hostname during
a migration (either source or target). Modify the query migration
function to check for the presence and set the field for future
consumers to determine which of 3 conditions is being met (NULL,
present and set to "", or present and sent to something). These
correspond to qemu commit id '4af245dc3' which added support to
default the value to "" and allow setting (or resetting) to ""
in order to disable. This reset option allows libvirt to properly
use the tls-creds and tls-hostname parameters.

Modify code paths that either allocate or use stack space in order
to call qemuMigrationParamsClear or qemuMigrationParamsFree for cleanup.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

The 'mon' argument validity is checked in the QEMU_CHECK_MONITOR for the
following functions, so they don't need the NONNULL on their prototype:

qemuMonitorUpdateVideoMemorySize
qemuMonitorUpdateVideoVram64Size
qemuMonitorGetAllBlockStatsInfo
qemuMonitorBlockStatsUpdateCapacity
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Commit id '85af0b80' added a 'timeout' as the 4th parameter to
qemuMonitorOpen, but neglected to update the ATTRIBUTE_NONNULL(4)
to be (5) for the cb parameter.

qemuMonitorGetGuestCPU can now optionally create CPU data from
filtered-features in addition to feature-words.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

There were couple of reports on the list (e.g. [1]) that guests
with huge amounts of RAM are unable to start because libvirt
kills qemu in the initialization phase. The problem is that if
guest is configured to use hugepages kernel has to zero them all
out before handing over to qemu process. For instance, 402GiB
worth of 1GiB pages took around 105 seconds (~3.8GiB/s). Since we
do not want to make the timeout for connecting to monitor
configurable, we have to teach libvirt to count with this
fact. This commit implements "1s per each 1GiB of RAM" approach
as suggested here [2].

1: https://www.redhat.com/archives/libvir-list/2017-March/msg00373.html
2: https://www.redhat.com/archives/libvir-list/2017-March/msg00405.htmlSigned-off-by: NMichal Privoznik <mprivozn@redhat.com>

The static CPU model expansion is designed to return only canonical
names of all CPU properties. To maintain backwards compatibility libvirt
is stuck with different spelling of some of the features, but we need to
use the full expansion to get the additional spellings. In addition to
returning all spelling variants for all properties the full expansion
will contain properties which are not guaranteed to be migration
compatible. Thus, we need to combine both expansions. First we need to
call the static expansion to limit the result to migratable properties.
Then we can use the result of the static expansion as an input to the
full expansion to get both canonical names and their aliases.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

While query-cpu-model-expansion returns only boolean features on s390,
but x86_64 reports some integer and string properties which we are
interested in.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

query-cpu-model-expansion is used to get a list of features for a given cpu
model name or to get the model and features of the host hardware/environment
as seen by Qemu/kvm.
Signed-off-by: NCollin L. Walling <walling@linux.vnet.ibm.com>
Signed-off-by: NJason J. Herne <jjherne@linux.vnet.ibm.com>