If a disk is unplugged and then the user tries to delete a checkpoint
the code would try to use NULL node name as it was not checked.

Fix this by fetching the whole disk definition object and verifying it
was found.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Lookup the whole disk definition rather than just the node name.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Upcoming patches will also use the domain disk definition. Rename disk
to chkdisk for clarity.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Upcoming patches will also use the domain disk definition. Rename disk
to chkdisk for clarity.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

qemuCheckpointDiscard is a massive function that can be separated into
smaller bits. Extract the part that actually modifies the disk from the
metadata handling.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Avoid two computations by using a boolean.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

qemu-5.0 will drop pre pc-1.0 machine types. Remove them from our
faked capabilities test suite. If a feature depends on a machine type it
shall be tested with real data and not with this hack.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

None of the tests depend on anything that the machine type would
influence. This will allow us to drop the very old machine type from the
non-real-data tests. If something depends on the machine type it should
be tested with real data rather than this hack.

Note that these tests are run only in the XML->XML suite because the
XML->argv suite doesn't work with the network driver.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

None of the tests depend on anything that the machine type would
influence. This will allow us to drop the very old machine type from the
non-real-data tests. If something depends on the machine type it should
be tested with real data rather than this hack.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Based on upstream commit 3e08b2b9cb64. This version already dropped the
pre-historic machine types and supports only machine types starting from
'pc-1.0'.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Remove the old machine type which will be dropped in the upcomming
qemu-5.0 release from tests used against the most recent capabilities
data.

None of the modified tests really cares about the actual machine type.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Skip the step of adding all of the fake machine types which are required
for the legacy tests in case when we are testing with real capabilities.

Faking any data in the real capabilities undermines the point of testing
with real capabilities.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Test code will need to know whether the virQEMUCaps object contains any
machine types already. Add a helper and expose it via 'qemu_capspriv.h'.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

The previous approac of just purging the alias combined with the fact
that we filled in fake machine types in the test data meant that if a
test case used an alias machine type such as 'pc' or 'q35' it would not
properly resolve to the actual data returned by qemu.

This started to be a problem since the CPU driver now looks at the
default CPU reported with the machine type.

This patch replaces the original approach of just removing the alias by
replacing it with a copy of the machine type data which the type would
alias to. This means that we are using the real data while we don't
modify the test output after every qemu upgrade.

Additionally this change will allow us to drop adding the fake machine
types later.

The test fallout is from actually excercising the CPU driver with
actual data.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Separate out the internals as they will become more complex soon.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Every supported qemu is able to return the list of machine types it
supports so we can start validating it against that list. The advantage
is a better error message, and the change will also prevent having stale
test data.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Enumerate all missing machine types for all missing architectures for
the fake capabilities used in many existing tests. This will allow
stricter validation whether qemu actually supports given machine type
since we already have some behaviour dependant on the actual machine
type.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Use the 's390-ccw-virtio' machine type which is actually supported by
the qemu we gathered the test data from.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

This machine type comes from downstream ubuntu 15.10. Replace it with a
somewhat equivalent qemu-2.3 machine type as we do have test data for
that.

The change allows the CPU code to pick a proper default CPU in the
'-latest' cases.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Use 'versatilepb' instead of a fake 'non-virt' machine type.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Use 'pc' instead of ancient 'rhel-6.1'.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

For testing with synthetic capabilities we pre-fill the qemu
capabilities with some machine types. Historically there were two arrays
for KVM and TCG but that's not necessary. Make both instances of x86_64
data share the same array as the other architectures do.

This will later on simplify filling in all the other machine types which
are required for the test suite.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

The tests prefixed default-video* were enabled only for the xml2xml
testing and used impossible configurations.

Enable them for xml2argv testing fix them:
1) aarch64: remove pointless cpu mode
2) s390x: remove pointless cpu and use existing machine type
3) riscv: remove pointless cpu
4) x86: remove pointless cpu and use existing machine type
5) ppc65: use correct machine type and enable USB
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

XML->XML testing uses DO_TEST_CAPS_LATEST so use it also for the
XML->argv testing. Additionally use the same more modern machine type
in both tests.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Mirror what's done in the xml2argv test and use recent capabilities.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

The data is tested against the latest qemu binaries so we should use the
proper architecture. Also the test is used against data from qemu 1.5.3
and thus we should use a machine type that qemu supported.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Debian/Ubuntu linkers are more strict that other distros requiring glib
to be linked explicitly.

macOS needs -export-dynamic instead of -Wl,--export-dynamic
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Similarly to 510d154a we need to prevent
doing too deeply nested backing chains and reject them with a sane error
message.

Add a loop to go through the snapshots prior to attempting actually
creating them to prevent some possible inconsistent scenarios.

We don't need to do it when reusing backing chains as we'll be
re-detecting the backing chain in that case anyways.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Don't adopt the backing store data when reusing images provided by the
user. This will force a backing chain re-probe as users might have
passed in something unexpected in the overlay where our view of the
backing chain would not correspond.

This is done only for inactive snapshots as there we have way less
verification.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Currently the libvirt Go modules are accessed by applications using
their github repository URLs. This is undesirable as we don't want
applications to have a direct dependancy on a specific source repo
location. We want to enable applications to use the Go packages via
the libvirt.org namespace.

When you do "go get libvirt.org/libvirt-go", the Go client will do an
HTTPS request to that URL, and parse the HTML content to look for a
<meta> tag which tells it where to the find the GIT repository.

   https://golang.org/cmd/go/#hdr-Remote_import_paths

This adds two pages to support this Go module resolution. They are
not linked from anywhere as we don't expect users to actually look
at them. If someone does happen upon them, there's some boilerplate
text to send them off to godoc.org for API documentation.

Since the pages we're adding have a .html extension, we will also
use a small apache config tweak on the server

    RewriteEngine on
    RewriteRule ^/libvirt-go$ /libvirt-go.html [L]
    RewriteRule ^/libvirt-go-xml$ /libvirt-go-xml.html [L]
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The previous "QEMU shim" proof of concept was taking an approach of only
caring about initial spawning of the QEMU process. It was then
registered with the libvirtd daemon who took over management of it. The
intent was that later libvirtd would be refactored so that the shim
retained control over the QEMU monitor and libvirt just forwarded APIs
to each shim as needed. This forwarding of APIs would require quite alot
of significant refactoring of libvirtd to achieve.

This impl thus takes a quite different approach, explicitly deciding to
keep the VMs completely separate from those seen & managed by libvirtd.
Instead it uses the new "qemu:///embed" URI scheme to embed the entire
QEMU driver in the shim, running with a custom root directory.

Once the driver is initialization, the shim starts a VM and then waits
to shutdown automatically when QEMU shuts down, or should kill QEMU if
it is terminated itself. This ought to use the AUTO_DESTROY feature but
that is not yet available in embedded mode, so we rely on installing a
few signal handlers to gracefully kill QEMU. This isn't reliable if
we crash of course, but you can restart with the same root dir.

Note this program does not expose any way to manage the QEMU process,
since there's no RPC interface enabled. It merely starts the VM and
cleans up when the guest shuts down at the end. This program is
installed to /usr/bin/virt-qemu-run enabling direct use by end users.
Most use cases will probably want to integrate the concept directly
into their respective application codebases. This standalone binary
serves as a nice demo though, and also provides a way to measure
performance of the startup process quite simply.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

This enables support for running the secret driver embedded to the
calling application process using a URI:

   secret:///embed?root=/some/path

When using the embedded mode with a root=/var/tmp/embed, the
driver will use the following paths:

       configDir: /var/tmp/embed/etc/secrets
        stateDir: /var/tmp/embed/run/secrets

These are identical whether the embedded driver is privileged
or unprivileged.

This compares with the system instance which uses

       configDir: /etc/libvirt/secrets
        stateDir: /var/lib/libvirt/secrets

When an embedded instance of the secret driver is open, any other
embedded drivers will automatically use the embedded secret driver.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

This enables support for running QEMU embedded to the calling
application process using a URI:

   qemu:///embed?root=/some/path

Note that it is important to keep the path reasonably short to
avoid risk of hitting the limit on UNIX socket path names
which is 108 characters.

When using the embedded mode with a root=/var/tmp/embed, the
driver will use the following paths:

                logDir: /var/tmp/embed/log/qemu
           swtpmLogDir: /var/tmp/embed/log/swtpm
         configBaseDir: /var/tmp/embed/etc/qemu
              stateDir: /var/tmp/embed/run/qemu
         swtpmStateDir: /var/tmp/embed/run/swtpm
              cacheDir: /var/tmp/embed/cache/qemu
                libDir: /var/tmp/embed/lib/qemu
       swtpmStorageDir: /var/tmp/embed/lib/swtpm
 defaultTLSx509certdir: /var/tmp/embed/etc/pki/qemu

These are identical whether the embedded driver is privileged
or unprivileged.

This compares with the system instance which uses

                logDir: /var/log/libvirt/qemu
           swtpmLogDir: /var/log/swtpm/libvirt/qemu
         configBaseDir: /etc/libvirt/qemu
              stateDir: /run/libvirt/qemu
         swtpmStateDir: /run/libvirt/qemu/swtpm
              cacheDir: /var/cache/libvirt/qemu
                libDir: /var/lib/libvirt/qemu
       swtpmStorageDir: /var/lib/libvirt/swtpm
 defaultTLSx509certdir: /etc/pki/qemu

At this time all features present in the QEMU driver are available when
running in embedded mode, availability matching whether the embedded
driver is privileged or unprivileged.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The driver URI scheme:

  "$drivername:///embed?root=/some/path"

enables a new way to use the drivers by embedding them directly in the
calling process. To use this the process must have a thread running the
libvirt event loop. This URI will then cause libvirt to dynamically load
the driver module and call its global initialization function. This
syntax is applicable to any driver, but only those will have been
modified to support a custom root directory and embed URI path will
successfully open.

The application can now make normal libvirt API calls which are all
serviced in-process with no RPC layer involved.

It is required to specify an explicit root directory, and locks will be
acquired on this directory to avoid conflicting with another app that
might accidentally pick the same directory.

Use of '/' is not explicitly forbidden, but note that the file layout
used underneath the embedded driver root does not match the file
layout used by system/session mode drivers. So this cannot be used as
a backdoor to interact with, or fake, the system/session mode drivers.

Libvirt will create arbitrary files underneath this root directory. The
root directory can be kept untouched across connection open attempts if
the application needs persistence. The application is responsible for
purging everything underneath this root directory when finally no longer
required.

Even when a virt driver is used in embedded mode, it is still possible
for it to in turn use functionality that calls out to other secondary
drivers in libvirtd. For example an embedded instance of QEMU can open
the network, secret or storage drivers in the system libvirtd.

That said, the application would typically want to at least open an
embedded secret driver ("secret:///embed?root=/some/path"). Note that
multiple different embedded drivers can use the same root prefix and
co-operate just as they would inside a normal libvirtd daemon.

A key thing to note is that for this to work, the application that links
to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that
symbols from libvirt.so are exported & thus available to the dynamically
loaded driver module. If libvirt.so itself was dynamically loaded then
RTLD_GLOBAL must be passed to dlopen().
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The intent here is to allow the virt drivers to be run directly embedded
in an arbitrary process without interfering with libvirtd. To achieve
this they need to store all their configuration & state in a separate
directory tree from the main system or session libvirtd instances.

This can be useful for doing testing of the virt drivers in "make check"
without interfering with the user's own libvirtd instances.

It can also be used for applications using KVM/QEMU as a piece of
infrastructure to build an service, rather than for general purpose
OS hosting. A long standing example is libguestfs, which would prefer
if its temporary VMs did show up in the main libvirtd VM list, because
this confuses apps such as OpenStack Nova. A more recent example would
be Kata which is using KVM as a technology to build containers.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

If a domain is configured to have an egl-headless display and a virtio
video device, virgl will be enabled automatically within the guest, even
if the video device is configured with accel3d='no'.

In this case we should explicitly pass 'virgl=off' to qemu.

See https://bugzilla.redhat.com/show_bug.cgi?id=1791236 for more
information.
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>