Do decisions based on the configuration of the job rather than the data
stored with the disk.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Use the stored job name rather than passing in the disk alias when
referring to the job which allows the same code to work also when
-blockdev will be used.

Note that this API does not require the change to use 'query-job' as it
will ever only work with blockjobs bound to disks due to the arguments
which allow only referring to a disk. For the disk-less jobs we'll need
to add a separate API later.

The change to qemuMonitorGetBlockJobInfo is required as the API was
stripping the 'drive-' prefix when returning the data which is not
desired any more.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virDomainSnapshotFindByName(list, NULL) should return NULL, rather
than the internal-use-only metaroot.  Most existing callers pass in a
non-NULL name; the few external callers that don't are immediately
calling virDomainMomentSetParent (which indeed needs the metaroot
rather than NULL if the parent name is NULL); but as the leaky
abstraction is ugly, it is worth instead making
virDomainMomentSetParent static and adding a new function for
resolving the parent link of a brand new moment within its list.  The
existing external uses of virDomainMomentSetParent always succeed
(either the new moment has parent_name of NULL to become a new root,
or has parent_name set to a strdup of the previous current moment);
hence, our new function does not need a return value (but it still has
a VIR_WARN in case future uses break our assumptions about failure
being impossible).

Missed when commit 02c4e24d refactored things to attempt to remove
direct metaroot manipulations out of the qemu and test drivers into
internal-only details, and made more obvious when commit dc8d3dc6
factored it out into a separate file.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Some VM configurations may result in a large number of threads created by
the associated qemu process which can exceed the system default limit. The
maximum number of threads allowed per process is controlled by the pids
cgroup controller and is set to 16k when creating VMs with systemd's
machined service. The maximum number of threads per process is recorded
in the pids.max file under the machine's pids controller cgroup hierarchy,
e.g.

$cgrp-mnt/pids/machine.slice/machine-qemu\\x2d1\\x2dtest.scope/pids.max

Maximum threads per process is controlled with the TasksMax property of
the systemd scope for the machine. This patch adds an option to qemu.conf
which can be used to override the maximum number of threads allowed per
qemu process. If the value of option is greater than zero, it will be set
in the TasksMax property of the machine's scope after creating the machine.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

It's already dereffed in the initialization and shouldn't be NULL
unless virJSONValueArraySize after a virJSONValueObjectGetArray
could return a NULL data entry.

Found by Coverity
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>
ACKed-by: NPeter Krempa <pkrempa@redhat.com>

Report in logs when we don't find existing block job data and create it
just to handle the job.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

While this function does start a block job in case when we'd not be able
to get our internal data for it, the handler sets the job state to
QEMU_BLOCKJOB_STATE_RUNNING anyways, thus qemuBlockJobStartupFinalize
would just unref the job.

Since the other usage of qemuBlockJobStartupFinalize in the other part
of the event handler was a bug replace this one anyways even if it would
not cause problems.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The block job event handler qemuProcessHandleBlockJob looks at the block
job data to see whether the job requires synchronous handling. Since the
block job event may arrive before we continue the job handling (if the
job has no data to copy) we could hit the state when the job is still
set as QEMU_BLOCKJOB_STATE_NEW (as we move it to the
QEMU_BLOCKJOB_STATE_RUNNING state only after returning from monitor).

If the event handler uses qemuBlockJobStartupFinalize it would
unregister and free the job. Thankfully this is not a big problem for
legacy blockjobs as we don't need much data for them but since we'd
re-instantiate the job data structure we'd report wrong job type for
active commit as qemu reports it as a regular commit job.

Fix it by not using qemuBlockJobStartupFinalize function in
qemuProcessHandleBlockJob as it is not starting the job anyways.

https://bugzilla.redhat.com/show_bug.cgi?id=1721375Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Commit 5ff46aaa added a new parameter but neglected to fix the NONNULL
declarations.
Reported-by: NJulio Faracco <jcfaracco@gmail.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

When removing the disk fronted while any block job is still active we
need to transfer the ownership of the backing chain to the job itself as
the job still holds the reference to the chain members and thus attempts
to remove them would fail.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

In cases when the disk frontend was unplugged while a blockjob was
running the blockjob inherits the backing chain. When the blockjob is
then terminated we need to unplug the chain as it will not be used any
more.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The PR manager is a property of the format layer in qemu so we need to
be able to track it also in the chains of orphaned block jobs.

Add a helper for qemu to look also into the blockjob state.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

When the guest unplugs the disk frontend libvirt is responsible for
deleting the backend. Since a blockjob may still have a reference to the
backing chain when it is running we'll have to store the metadata for
the unplugged disk for future reference.

This patch adds 'chain' and 'mirrorChain' fields to 'qemuBlockJobData'
to keep them around with the job along with status XML machinery and
tests. Later patches will then add code to change the ownership of the
chain when unplugging the disk backend.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Refresh the state of the jobs and process any events that might have
happened while libvirt was not running.

The job state processing requires some care to figure out if a job
needs to be bumped.

For any invalid job try doing our best to cancel it.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Add the infrastructure to handle block job events in the -blockdev era.

Some complexity is required as qemu does not bother to notify whether
the job was concluded successfully or failed. Thus it's necessary to
re-query the monitor.

To minimize the possibility of stuck jobs save the state into the XML
prior to handling everything so that the reconnect code can potentially
continue with the cleanup.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Add support for handling the event either synchronously or
asynchronously using the event thread.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

With blockdev we'll need to use the JOB_STATUS_CHANGE so gate the old
events by the blockdev capability.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

This new state is entered when qemu finished the job but libvirt does
not know whether it was successful or not.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Now that the blockjob handling code deals with the status XML we don't
need to save it explicitly when starting blockjobs.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Now that block job data is stored in the status XML portion we need to
make sure that everything which changes the state also saves the status
XML. The job registering function is used while parsing the status XML
so in that case we need to skip the XML saving.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

We need to store the block job state in the status XML so that we can
properly recover any data when reconnecting after startup and also in
the end to be able to do any transition of the backing chain that
happened while libvirt was not connected to the monitor.

First step is to note the name, type, state and corresponding disk into
the status XML.

We also need to make sure that a broken blockjob does not make libvirt
lose the VM, thus many of the errors just mark the job as invalid.
Later on we'll cancel all invalid jobs.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The job data saved in the XML may be partially invalid e.g. if something
is missing. To prevent losing a domain with such a job add a flag to the
job data so that job APIs can ignore such a job and we can just cancel
it.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

When parsing the status XML we need to register all existing jobs.
Export the functions so that they are usable in other modules.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Later on we'll format these values into the status XML so the from/to
string functions will come handy. The implementation also notes that
these will be used in the status XML to avoid somebody changing the
values.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Add the job structure to the table when instantiating a new job and
remove it when it terminates/fails.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Block jobs currently belong to disks only so we can look up the block
job data for them in the corresponding disks. This won't be the case
when using blockdev as certain jobs don't even correspond to a disk and
most of them can run on a part of the backing chain.

Add a global table of blockjobs which can be used to look up the data
for the blockjobs when the job events need to be processed.

The table is a hash table organized by job name and has a reference to
the job. New and running jobs will later be added to this table.
Reference counting will allow to reap job state for synchronous callers.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The legacy job handler does not look at the old job state so we can
update it earlier.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

There's no need to do it if the job is not completed. The new helper
allows to do this with much less hassle in the correct place.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Rename and move qemuBlockJobTerminate to qemuBlockJobUnregister and
separate bits from qemuBlockJobDiskNew which register the job with the
disk. This creates an unified interface for other APIs to use.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Simplify error paths.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Similarly to qemuDomainSaveStatus add a helper to save the config XML
named qemuDomainSaveConfig.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Rename qemuDomainObjSaveJob and create a wrapper for it which does not
require 'driver' to be passed and export it so that other palces can
easily save the status XML without having to invoke virDomainSaveStatus
which has unpleasing parameters.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

QEMU allows us to create storage on certain network protocols which
allow image creation through their API. Wire up the generator for using
it with libvirt as well as for local files.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

'blockdev-add' allows us to use qemu to format images to our desired
format. This patch implements helpers which convert a
virStorageSourcePtr into JSON objects describing the required
configuration.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

To allow using -blockdev with blockjobs QEMU needs to reopen files in
read-write mode when modifying the backing chain. To achieve this we
need to use 'auto-read-only' for the backing files rather than the
normal 'read-only' property. That way qemu knows that the files need to
be reopened.

Note that the format drivers (e.g. qcow2) are still opened with the
read-only property enabled when being a member of the backing chain
since they are supposed to be immutable unless a block job is started.

QEMU v4.0 (since commit 23dece19da4) allows also dynamic behaviour for
auto-read-only which allows us to use sVirt as we only grant write
permissions to files when doing a blockjob.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

To allow reusing the formatters in the code for creating JSON properties
for 'blockdev-create' we need to create everything except the 'driver'
attribute.

Use the new helper virJSONValueObjectPrependString to put the driver at
the same place so that we don't change any output.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

When formatting new qcow2 images we need to provide the backing store
string which should not contain any authentication or irrelevant data.

Add a flag for qemuBlockStorageSourceGetBackendProps which allows to
skip the irrelevant data.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The 'blockdev-create' starts a job which creates a storage volume using
the given protocol or formats an existing (added) volume with one of the
supported storage formats.

This patch adds the monitor interaction bits.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>