- Remove all qemu emulators
- Restart libvirtd
- Install qemu emulators
- Call 'virsh version' -> errors

The only thing that will force the qemu driver to refresh it's cached
capablities info is an explict API call to GetCapabilities.

However in the case when the initial caps lookup at driver connect didn't
find a single qemu emulator to poll, the driver is effectively useless
and really can't do anything until it's populated some qemu capabilities
info.

With the above steps, the user would have to either know about the
magic refresh capabilities call, or restart libvirtd to pick up the
changes.

Instead, this patch changes things so that every time a part of th
driver requests access to capabilities info, check to see if
we've previously seen any emulators. If not, force a refresh.

In the case of 'still no emulators found', this is still very quick, so
I can't think of a downside.

https://bugzilla.redhat.com/show_bug.cgi?id=1000116
(cherry picked from commit 95546c43)

If VM is configured with many devices(including passthrough devices)
and large memory, libvirtd will take seconds(in the worst case) to
wait for monitor. In this period the qemu process may run on any
PCPU though I intend to pin emulator to the specified PCPU in xml
configuration.

Actually qemu process takes high cpu usage during vm startup.
So this is not the strict CPU isolation in this case.
Signed-off-by: NZhou yimin <zhouyimin@huawei.com>
(cherry picked from commit 411cea63)

If the memory mode is specified as 'strict' and with one node, we
get the following error when starting domain.

error: Unable to write to '$cgroup_path/cpuset.mems': Device or resource busy

XML is configured with numatune as follows:
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>

It's broken by Commit 411cea63
which moved qemuSetupCgroupForEmulator() before setting cpuset.mems
in qemuSetupCgroupPostInit.

Directory '$cgroup_path/emulator/' is created in qemuSetupCgroupForEmulator.
But '$cgroup_path/emulator/cpuset.mems' it not set and has a default value
(all nodes, such as 0-1). Then we setup '$cgroup_path/cpuset.mems' to the
nodemask (in this case it's '0') in qemuSetupCgroupPostInit. It must fail.

This patch makes '$cgroup_path/emulator/cpuset.mems' is set before
'$cgroup_path/cpuset.mems'. The action is similar with that in
qemuDomainSetNumaParamsLive.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>
(cherry picked from commit c6e90248)

The ACL check didn't check the VIR_DOMAIN_XML_SECURE flag and the
appropriate permission for it. Found via code inspection while fixing
permissions for save images.

(cherry picked from commit b347c0c2)

The ACL check didn't check the VIR_DOMAIN_XML_SECURE flag and the
appropriate permission for it.

(cherry picked from commit 03c3c0c8)

Commit ca91ba78 moved qemuSetupDiskCgroup into the qemuDomainPrepareDisk
helper, but failed to call it for usb disks.

https://bugzilla.redhat.com/show_bug.cgi?id=1175668`
(cherry picked from commit 1cddf000)
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Avoid leaving the domain locked on a failed ACL check in
qemuDomainMigratePerform() and qemuDomainMigrateFinish2().

Introduced in commit abf75aea (Add ACL checks into the QEMU driver).

(cherry picked from commit 2bdcd29c)

A logic bug in qemuConnectGetAllDomainStats makes the code mark the
monitor as available when qemuDomainObjBeginJob fails, instead of when
it succeeds, as the correct flow requires.

This patch fixes the check and updates the code documentation
accordingly.

Broken by commit 57023c0a.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>
(cherry picked from commit cb104ef7)
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

When user doesn't have read access on one of the domains he requested,
the for loop could exit abruptly or continue and override pointer which
pointed to locked object.

This patch fixed two issues at once.  One is that domflags might have
had QEMU_DOMAIN_STATS_HAVE_JOB even when there was no job started (this
is fixed by doing domflags |= QEMU_DOMAIN_STATS_HAVE_JOB only when the
job was acquired and cleaning domflags on every start of the loop.
Second one is that the domain is kept locked when
virConnectGetAllDomainStatsCheckACL() fails and continues the loop when
it didn't end.  Adding a simple virObjectUnlock() and clearing the
pointer ought to do.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 57023c0a)
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

As discussed on the upstream list, it's better not to make this
kind of predictions in libvirt. It may happen that qemu learns
how to enable OVMF on other architectures too and we shouldn't
try to chase that.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 36148120)

Currently, we are whitelisting architectures, that we know how to run
OVMF on. So far, only x86_64 was enabled. However, looking at qemu
code, the same commandline can be used to enable OVMF for armv7l and
aarch64.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 6d8054b6)

Since QEMU 1.2.0, we switched to QMP probing instead of parsing -help
(and other commands, such as -cpu ?) output. However, if QMP probing
failed, we still tried starting QEMU with various options and parsing
the output, which was guaranteed to fail because the output changed.
Let's just refuse parsing -help for QEMU >= 1.2.0.

https://bugzilla.redhat.com/show_bug.cgi?id=1160318Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
(cherry picked from commit ae3e29e6)

Conflicts:
	tests/qemuhelptest.c

We used to set migration capabilities only when a user asked for them in
flags. This is fine when migration succeeds since the QEMU process is
killed in the end but in case migration fails or if it's cancelled, some
capabilities may remain turned on with no way to turn them off. To fix
that, migration capabilities have to be turned on if requested but
explicitly turned off in case they were not requested but QEMU supports
them.

https://bugzilla.redhat.com/show_bug.cgi?id=1163953Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
(cherry picked from commit ab393383)

Commit 6e5c79a1 tried to fix deadlock between nwfilter{Define,Undefine}
and starting of guest, but this same deadlock exists for
updating/attaching network device to domain.

The deadlock was introduced by removing global QEMU driver lock because
nwfilter was counting on this lock and ensure that all driver locks are
locked inside of nwfilter{Define,Undefine}.

This patch extends usage of virNWFilterReadLockFilterUpdates to prevent
the deadlock for all possible paths in QEMU driver. LXC and UML drivers
still have global lock.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1143780Signed-off-by: NPavel Hrdina <phrdina@redhat.com>
(cherry picked from commit 41127244)

In one of my previous patches (3a3c3780) I've tried to fix the
problem of nvram path disappearing on a domain that's been
started and shut down again. I fixed this by explicitly saving
domain's config file.  However, I did a bit of clumsy without
realizing we have a transient domains for which we don't save the
config file. Hence, any domain using UEFI became persistent.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 54ddc08d)

Use the device type name if we know it instead of its number,
even if we can't hotplug it:
qemuMonitorJSONAttachCharDevCommand:6094 : operation failed: Unsupported
char device type '10'

(cherry picked from commit cce8e5f7)

https://bugzilla.redhat.com/show_bug.cgi?id=1160084

As of b6d4dad1 (1.2.5) libvirt keeps track if domain disks have been
frozen. However, this falls into that set of information which don't
survive domain restart. Therefore, we need to clear the flag upon some
state transitions. Moreover, once we clear the flag we must update the
status file too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 6ea54769)

In qemuMigrationFinish mig->nbd can not be initialized by
qemuMigrationEatCookie without the QEMU_MIGRATION_COOKIE_NBD flag.
That causes qemuMigrationStopNBDServer to return early without
stopping the NBD server properly.
Signed-off-by: NWeiwei Li <nuonuoli@tencent.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>
(cherry picked from commit c3012a02)

When daemon is killed right in the middle of probing a qemu binary for
its capabilities, the qemu process is left running.  Next time the
daemon is starting, it cannot start the probing qemu process because the
one that's already running does have the pidfile flock()'d.
Reported-by: NWang Yufei <james.wangyufei@huawei.com>
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 0ed1b55b)

Particularly in qemuBuildNumaArgStr(), there was a need for the advice
due to memory backing, which needs to know the nodeset it will be pinned
to.  With newer qemu this caused the following error when starting
domain:

  error: internal error: Advice from numad is needed in case of
  automatic numa placement

even when starting perfectly valid domain, e.g.:

  ...
  <vcpu placement='auto'>4</vcpu>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>
  <cpu>
    <numa>
      <cell id='0' cpus='0' memory='524288'/>
      <cell id='1' cpus='1' memory='524288'/>
    </numa>
  </cpu>
  ...

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1138545Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 11a48758)

commit 3e1e16aa (Use a port from the
migration range for NBD as well) changed ndb port allocation from
remotePorts to migrationPorts, but did not change the port releasing
process, which makes an error when migrating several times (above 64):
error: internal error: Unable to find an unused port in range
'migration' (49152-49215)

https://bugzilla.redhat.com/show_bug.cgi?id=1159245Signed-off-by: NWeiwei Li <nuonuoli@tencent.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>
(cherry picked from commit be598c5f)

https://bugzilla.redhat.com/show_bug.cgi?id=1140981 reports that
the qemu-kvm shipped as part of RHEL 7.0 intentionally[1] cripples
block jobs by removing the 'block-stream' QMP command, while still
leaving 'block-job-cancel' as an unusable no-op.  Meanwhile, we
already had existing code that checked whether block jobs were
completely missing (such as qemu 0.15), old style (cancel is
synchronous, and all commands spelled with '_'), or new style
(cancel is asynchronous, and all commands spelled with '-'), and
used that three-way probe to give decent error messages.  At the
time that code was added, all existing qemu versions fell in one
of three buckets, and the code was using the presence of
'block-job-cancel' as the witness of which of the three buckets.
But now that RHEL qemu has shipped with intentionally crippled
'block-stream', we have a fourth bucket, which results in ugly
error messages when trying 'virsh blockpull':

 error: Requested operation is not valid: Command 'block-stream' is not found

In reality, the fourth bucket should be treated the same as the
first bucket (no block job support); we can do that by realizing
that no existing build of qemu has working block-stream while
lacking block-job-cancel, so it is easiest to change our witness
to the command that starts a job rather than ends one.  We still
act correctly regarding command spelling and whether cancel is
asynchronous.  And on crippled RHEL builds, we now get the desired:

 error: unsupported configuration: block jobs not supported with this qemu binary

[1] The intentional cripple is limited to qemu-kvm of RHEL; when using
qemu-kvm-rhev of RHEV, block job functionality is supported.  Don't ask
me to explain the "why" behind it all - I'm just dealing with fallout
from someone else's decision.

* src/qemu/qemu_capabilities.h (QEMU_CAPS_BLOCKJOB_SYNC): Tweak comment.
* src/qemu/qemu_capabilities.c (virQEMUCapsCommands): Look for stream
rather than cancel when determining the flavor of block jobs supported.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 00331bfb)

Not every error message from qemu-ga has to have the 'class' field
filled out. For instance, I've seen this error message lately:

  qemuAgentCheckError:1047 : unable to execute QEMU agent command \
  {"execute":"guest-set-time"}: \
  {"error":{"desc":"Invalid parameter type, expected: integer"}}

However, this got translated into rather generic error message:

  internal error: unable to execute QEMU agent command
  'guest-set-time': unknown QEMU command error

So we've dropped better error message in favor of a generic one.
This is due to our code which expects 'class' which is not
present here.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit b7fe5a65)

https://bugzilla.redhat.com/show_bug.cgi?id=956506 documents that
given a domain where an internal snapshot parent has an external
snapshot child, we lacked a safety check when trying to use the
--children-only option to snapshot-delete:

$ virsh start dom
$ virsh snapshot-create-as dom internal
$ virsh snapshot-create-as dom external --disk-only
$ virsh snapshot-delete dom external
error: Failed to delete snapshot external
error: unsupported configuration: deletion of 1 external disk snapshots not supported yet
$ virsh snapshot-delete dom internal --children
error: Failed to delete snapshot internal
error: unsupported configuration: deletion of 1 external disk snapshots not supported yet
$ virsh snapshot-delete dom internal --children-only
Domain snapshot internal children deleted

While I'd still like to see patches that actually do proper external
snapshot deletion, we should at least fix the inconsistency in the
meantime.  With this patch:

$ virsh snapshot-delete dom internal --children-only
error: Failed to delete snapshot internal
error: unsupported configuration: deletion of 1 external disk snapshots not supported yet

* src/qemu/qemu_driver.c (qemuDomainSnapshotDelete): Fix condition.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 2086a990)

Also consider whitespace only strings returned from the hook as empty
result.

(cherry picked from commit 19b1ee42)

The documentation for the restore hook states that returning an empty
XML is equivalent with copying the input. There was a bug in the code
checking the returned string by checking the string instead of the
contents. Use the new helper to check if the string is empty.

(cherry picked from commit e3867799)

https://bugzilla.redhat.com/show_bug.cgi?id=1141732

Introduced by commit id '8f76ad99' the logic to detach a scsi_host
device (SCSI or iSCSI) fails when attempting to remove the 'drive'
because as I found in my investigation - the DelDevice takes care of
that for us.

The investigation turned up commits to adjust the logic for the
qemuMonitorDelDevice and qemuMonitorDriveDel processing for interfaces
(commit id '81f76598'), disk bus=VIRTIO,SCSI,USB (commit id '0635785b'),
and chr devices (commit id '55b21f9b'), but nothing with the host devices.

This commit uses the model for the previous set of changes and applies
it to the hostdev path. The call to qemuDomainDetachHostSCSIDevice will
return to qemuDomainDetachThisHostDevice handling either the audit of
the failure or the wait for the removal and then call into
qemuDomainRemoveHostDevice for the event, removal from the domain hostdev
list, and audit of the removal similar to other paths.

NOTE: For now the 'conn' param to +qemuDomainDetachHostSCSIDevice is left
as ATTRIBUTE_UNUSED.  Removing requires a cascade of other changes to be
left for a future patch.

(cherry picked from commit d2774e54)

Commit 4882618e added the code that
requests driver cfg, but forgot to unref it.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 9661ac2f)

If we don't properly clean up all processes in the
machine-<vmname>.scope systemd won't remove the cgroup and subsequent vm
starts fail with

  'CreateMachine: File exists'

Additional processes can e.g. be added via

  echo $PID > /sys/fs/cgroup/systemd/machine.slice/machine-${VMNAME}.scope/tasks

but there are other cases like

  http://bugs.debian.org/761521

Invoke TerminateMachine to be on the safe side since systemd tracks the
cgroup anyway. This is a noop if all processes have terminated already.

(cherry picked from commit 4882618e)

virt-manager on Fedora sets up i686 hosts with "/usr/bin/qemu-kvm" emulator,
which in turn unconditionally execs qemu-system-x86_64 querying capabilities
then fails:

Error launching details: invalid argument: architecture from emulator 'x86_64' doesn't match given architecture 'i686'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/engine.py", line 748, in _show_vm_helper
    details = self._get_details_dialog(uri, vm.get_connkey())
  File "/usr/share/virt-manager/virtManager/engine.py", line 726, in _get_details_dialog
    obj = vmmDetails(conn.get_vm(connkey))
  File "/usr/share/virt-manager/virtManager/details.py", line 399, in __init__
    self.init_details()
  File "/usr/share/virt-manager/virtManager/details.py", line 784, in init_details
    domcaps = self.vm.get_domain_capabilities()
  File "/usr/share/virt-manager/virtManager/domain.py", line 518, in get_domain_capabilities
    self.get_xmlobj().os.machine, self.get_xmlobj().type)
  File "/usr/lib/python2.7/site-packages/libvirt.py", line 3492, in getDomainCapabilities
    if ret is None: raise libvirtError ('virConnectGetDomainCapabilities() failed', conn=self)
libvirtError: invalid argument: architecture from emulator 'x86_64' doesn't match given architecture 'i686'

Journal:

Oct 16 21:08:26 goatlord.localdomain libvirtd[1530]: invalid argument: architecture from emulator 'x86_64' doesn't match given architecture 'i686'

(cherry picked from commit afe8f420)

Right now when building the qemu command line, we try to do various
unconditional validations of the guest CPU against the host CPU. However
this checks are overly applied. The only time we should use the checks
are:

- The user requests host-model/host-passthrough, or

- When KVM is requsted. CPU features requested in TCG mode are always
  emulated by qemu and are independent of the host CPU, so no host CPU
  checks should be performed.

Right now if trying to specify a CPU for arm on an x86 host, it attempts
to do non-sensical validation and falls over.

Switch all the test cases that were intending to test CPU validation to
use KVM, so they continue to test the intended code.

Amend some aarch64 XML tests with a CPU model, to ensure things work
correctly.

(cherry picked from commit cf7fce8f2fd1c930f357fd4ff93ac35f38eb30c6)

Move the CPU mode/model handling to its own function. This is just
code movement and re-indentation.

(cherry picked from commit e1d872dc77c80d43036f928f83f560f2e9286148)

This removes the artificial and unnecessary restriction that
virDomainSetMaxDowntime() only be called while a migration is in
progress.

https://bugzilla.redhat.com/show_bug.cgi?id=1146618Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

FreeBSD's compiler complains that we shadow the symbol. Sigh.

s/devname/dev_name/

The current block stats code matched up the disk name with the actual
stats by the order in the data returned from qemu. This unfortunately
isn't right as qemu may return the disks in any order. Fix this by
returning a hash of stats and index them by the disk alias.

Commit fba6bc47 introduced the non-migratable invtsc feature,
breaking save/migration with host-model and host-passthrough.

On hosts with this feature present it was automatically included
in the CPU definition, regardless of QEMU support.

Commit de0aeafe stopped including it by default for host-model,
but failed to fix host-passthrough.

This commit ignores checking of CPU features with host-passthrough,
since we don't pass them to QEMU (only -cpu host is passed),
allowing domains using host-passthrough that were saved with
the broken version of libvirtd to be restored.

https://bugzilla.redhat.com/show_bug.cgi?id=1147584

Introduced by commit 0dce260c.

qemuDomainEventQueue was called with qemuDomainObjPrivatePtr instead
of virQEMUDriverPtr.

https://bugzilla.redhat.com/show_bug.cgi?id=1147494

Prompted by

   http://bugs.debian.org/761131

Fix the typo in VIR_DOMAIN_TUNABLE_CPU_EMULATORIN
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

For the new VIR_DOMAIN_EVENT_ID_TUNABLE event we have a bunch of
constants added

   VIR_DOMAIN_EVENT_CPUTUNE_<blah>
   VIR_DOMAIN_EVENT_BLKDEVIOTUNE_<blah>

This naming convention is bad for two reasons

  - There is no common prefix unique for the events to both
    relate them, and distinguish them from other event
    constants

  - The values associated with the constants were chosen
    to match the names used with virConnectGetAllDomainStats
    so having EVENT in the constant name is not applicable in
    that respect

This patch proposes renaming the constants to

    VIR_DOMAIN_TUNABLE_CPU_<blah>
    VIR_DOMAIN_TUNABLE_BLKDEV_<blah>

ie, given them a common VIR_DOMAIN_TUNABLE prefix.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>