Currently the libvirt Go modules are accessed by applications using
their github repository URLs. This is undesirable as we don't want
applications to have a direct dependancy on a specific source repo
location. We want to enable applications to use the Go packages via
the libvirt.org namespace.

When you do "go get libvirt.org/libvirt-go", the Go client will do an
HTTPS request to that URL, and parse the HTML content to look for a
<meta> tag which tells it where to the find the GIT repository.

   https://golang.org/cmd/go/#hdr-Remote_import_paths

This adds two pages to support this Go module resolution. They are
not linked from anywhere as we don't expect users to actually look
at them. If someone does happen upon them, there's some boilerplate
text to send them off to godoc.org for API documentation.

Since the pages we're adding have a .html extension, we will also
use a small apache config tweak on the server

    RewriteEngine on
    RewriteRule ^/libvirt-go$ /libvirt-go.html [L]
    RewriteRule ^/libvirt-go-xml$ /libvirt-go-xml.html [L]
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The previous "QEMU shim" proof of concept was taking an approach of only
caring about initial spawning of the QEMU process. It was then
registered with the libvirtd daemon who took over management of it. The
intent was that later libvirtd would be refactored so that the shim
retained control over the QEMU monitor and libvirt just forwarded APIs
to each shim as needed. This forwarding of APIs would require quite alot
of significant refactoring of libvirtd to achieve.

This impl thus takes a quite different approach, explicitly deciding to
keep the VMs completely separate from those seen & managed by libvirtd.
Instead it uses the new "qemu:///embed" URI scheme to embed the entire
QEMU driver in the shim, running with a custom root directory.

Once the driver is initialization, the shim starts a VM and then waits
to shutdown automatically when QEMU shuts down, or should kill QEMU if
it is terminated itself. This ought to use the AUTO_DESTROY feature but
that is not yet available in embedded mode, so we rely on installing a
few signal handlers to gracefully kill QEMU. This isn't reliable if
we crash of course, but you can restart with the same root dir.

Note this program does not expose any way to manage the QEMU process,
since there's no RPC interface enabled. It merely starts the VM and
cleans up when the guest shuts down at the end. This program is
installed to /usr/bin/virt-qemu-run enabling direct use by end users.
Most use cases will probably want to integrate the concept directly
into their respective application codebases. This standalone binary
serves as a nice demo though, and also provides a way to measure
performance of the startup process quite simply.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

This enables support for running the secret driver embedded to the
calling application process using a URI:

   secret:///embed?root=/some/path

When using the embedded mode with a root=/var/tmp/embed, the
driver will use the following paths:

       configDir: /var/tmp/embed/etc/secrets
        stateDir: /var/tmp/embed/run/secrets

These are identical whether the embedded driver is privileged
or unprivileged.

This compares with the system instance which uses

       configDir: /etc/libvirt/secrets
        stateDir: /var/lib/libvirt/secrets

When an embedded instance of the secret driver is open, any other
embedded drivers will automatically use the embedded secret driver.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

This enables support for running QEMU embedded to the calling
application process using a URI:

   qemu:///embed?root=/some/path

Note that it is important to keep the path reasonably short to
avoid risk of hitting the limit on UNIX socket path names
which is 108 characters.

When using the embedded mode with a root=/var/tmp/embed, the
driver will use the following paths:

                logDir: /var/tmp/embed/log/qemu
           swtpmLogDir: /var/tmp/embed/log/swtpm
         configBaseDir: /var/tmp/embed/etc/qemu
              stateDir: /var/tmp/embed/run/qemu
         swtpmStateDir: /var/tmp/embed/run/swtpm
              cacheDir: /var/tmp/embed/cache/qemu
                libDir: /var/tmp/embed/lib/qemu
       swtpmStorageDir: /var/tmp/embed/lib/swtpm
 defaultTLSx509certdir: /var/tmp/embed/etc/pki/qemu

These are identical whether the embedded driver is privileged
or unprivileged.

This compares with the system instance which uses

                logDir: /var/log/libvirt/qemu
           swtpmLogDir: /var/log/swtpm/libvirt/qemu
         configBaseDir: /etc/libvirt/qemu
              stateDir: /run/libvirt/qemu
         swtpmStateDir: /run/libvirt/qemu/swtpm
              cacheDir: /var/cache/libvirt/qemu
                libDir: /var/lib/libvirt/qemu
       swtpmStorageDir: /var/lib/libvirt/swtpm
 defaultTLSx509certdir: /etc/pki/qemu

At this time all features present in the QEMU driver are available when
running in embedded mode, availability matching whether the embedded
driver is privileged or unprivileged.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The driver URI scheme:

  "$drivername:///embed?root=/some/path"

enables a new way to use the drivers by embedding them directly in the
calling process. To use this the process must have a thread running the
libvirt event loop. This URI will then cause libvirt to dynamically load
the driver module and call its global initialization function. This
syntax is applicable to any driver, but only those will have been
modified to support a custom root directory and embed URI path will
successfully open.

The application can now make normal libvirt API calls which are all
serviced in-process with no RPC layer involved.

It is required to specify an explicit root directory, and locks will be
acquired on this directory to avoid conflicting with another app that
might accidentally pick the same directory.

Use of '/' is not explicitly forbidden, but note that the file layout
used underneath the embedded driver root does not match the file
layout used by system/session mode drivers. So this cannot be used as
a backdoor to interact with, or fake, the system/session mode drivers.

Libvirt will create arbitrary files underneath this root directory. The
root directory can be kept untouched across connection open attempts if
the application needs persistence. The application is responsible for
purging everything underneath this root directory when finally no longer
required.

Even when a virt driver is used in embedded mode, it is still possible
for it to in turn use functionality that calls out to other secondary
drivers in libvirtd. For example an embedded instance of QEMU can open
the network, secret or storage drivers in the system libvirtd.

That said, the application would typically want to at least open an
embedded secret driver ("secret:///embed?root=/some/path"). Note that
multiple different embedded drivers can use the same root prefix and
co-operate just as they would inside a normal libvirtd daemon.

A key thing to note is that for this to work, the application that links
to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that
symbols from libvirt.so are exported & thus available to the dynamically
loaded driver module. If libvirt.so itself was dynamically loaded then
RTLD_GLOBAL must be passed to dlopen().
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The intent here is to allow the virt drivers to be run directly embedded
in an arbitrary process without interfering with libvirtd. To achieve
this they need to store all their configuration & state in a separate
directory tree from the main system or session libvirtd instances.

This can be useful for doing testing of the virt drivers in "make check"
without interfering with the user's own libvirtd instances.

It can also be used for applications using KVM/QEMU as a piece of
infrastructure to build an service, rather than for general purpose
OS hosting. A long standing example is libguestfs, which would prefer
if its temporary VMs did show up in the main libvirtd VM list, because
this confuses apps such as OpenStack Nova. A more recent example would
be Kata which is using KVM as a technology to build containers.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

If a domain is configured to have an egl-headless display and a virtio
video device, virgl will be enabled automatically within the guest, even
if the video device is configured with accel3d='no'.

In this case we should explicitly pass 'virgl=off' to qemu.

See https://bugzilla.redhat.com/show_bug.cgi?id=1791236 for more
information.
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Since v4.2-rc0, QEMU introduced a builtin rng backend that uses
getrandom() syscall to generate random. Add it to libvirt with the
backend model 'builtin'.

https://bugzilla.redhat.com/show_bug.cgi?id=1785091Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

The 'builtin' rng backend model can be used as following:
  <rng model='virtio'>
    <backend model='builtin'/>
  </rng>
Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

For qemu object like rng-builtin, there are no properties after id
property. We should always set comma after object id. Otherwise it will
cause trailing comma on object:
    -object rng-builtin,id=ID,
Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

It is used to check if qemu is capable of rng-builtin object.

This object is added since qemu-4.2.0-rc0, commit 6c4e9d48.
Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Since v5.6.0-48-g270583ed we try to cache domain capabilities,
i.e. store filled virDomainCaps in a hash table in virQEMUCaps
for future use. However, there's a race condition in the way it's
implemented. We use virQEMUCapsGetDomainCapsCache() to obtain the
pointer to the hash table, then we search the hash table for
cached data and if none is found the domcaps is constructed and
put into the table. Problem is that this is all done without any
locking, so if there are two threads trying to do the same, one
will succeed and the other will fail inserting the data into the
table.

Also, the API looks a bit fishy - obtaining pointer to the hash
table is dangerous.

The solution is to use a mutex that guards the whole operation
with the hash table. Then, the API can be changes to return
virDomainCapsPtr directly.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1791790Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPeter Krempa <pkrempa@redhat.com>

When fixing [1] I've ran attached reproducer and had it spawn
1024 threads and query capabilities XML in each one of them. This
lead libvirtd to hit the RLIMIT_NOFILE limit which was kind of
expected. What wasn't expected was a subsequent segfault. It
happened because virCPUProbeHost failed and returned NULL. We've
taken the NULL and passed it to virCapabilitiesHostNUMARef()
which dereferenced it. Code inspection showed the same flas in
virQEMUDriverGetHostNUMACaps(), so I'm fixing both places.

1: https://bugzilla.redhat.com/show_bug.cgi?id=1791790Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPeter Krempa <pkrempa@redhat.com>

The virCapabilitiesGetNodeInfo() function has the usual return
value semantics for integeres: a negative value means an error,
zero or a positive value means success. However, the function
call done in virCPUProbeHost() doesn't check for the return value
accordingly.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPeter Krempa <pkrempa@redhat.com>

Don't use ERANGE as it doesn't make much sense in the error message.
Also point out that the reply from qemu was too large which is not
obvious from the original error:

 error: No complete monitor response found in 10485760 bytes: Numerical result out of range

The new message will read:

 error: internal error: QEMU monitor reply exceeds buffer size (10485760 bytes)
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

libvirt treats 'luks' images as raw+encryption. The logic in
qemuBlockStorageSourceCreateFormat skipped the creation if the requested
image was raw but didn't take into account the encryption.

This manifested itself e.g. when attempting to do a virsh blockcopy with
the following XML:

    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/tmp/enccpy'>
        <encryption format='luks'>
          <secret type='passphrase' uuid='0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f'/>
        </encryption>
      </source>
    </disk>

Where qemu would report the following error:

 unable to execute QEMU command 'blockdev-add': Volume is not in LUKS format

rather than actually formatting the image first.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Test that adding a duplicate entry is rejected properly. This also
allows to see the error message of the duplicate key addition in verbose
mode.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

If we get a user reporting this error message being shown it's pretty
useless in terms of actually debugging it since we don't know which hash
and which key are actually subject to the error.

This patch adds a new hash table callback which formats the
user-readable version of the hash key and reports it in the new message
which will look like:

"Duplicate hash table key 'blah'"

That way we will at least have an anchor point where to start the
search.

There are two special implementations of keys which are numeric so we
add specific printer functions for them.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Use the user-configured name of the bitmap when merging the appropriate
bitmaps for an incremental backup so that the user can see it as
configured. Additionally expose the default bitmap name if nothing is
configured.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Pass the exportname as configured when exporting the image via NBD and
fill it with the default if it's not configured.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

If users wish to use different name for exported disks or bitmaps
the new fields allow to do so. Additionally they also document the
current settings.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

When using blockdev configurations the 'device' argument of
'blockdev-commit' must correspond to the topmost node in the block node
graph. Libvirt didn't do this properly in case when 'copy_on_read'
option was enabled on the disk.

Use qemuDomainDiskGetTopNodename to fix it when calling block-commit.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

When using blockdev configurations the 'device' argument of
'blockdev-mirror' must correspond to the topmost node in the block node
graph. Libvirt didn't do this properly in case when 'copy_on_read'
option was enabled on the disk.

Use qemuDomainDiskGetTopNodename to fix it for the blockdev-mirror calls
in qemuDomainBlockCopy and the non-shared-storage migration.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

There are more places which require getting the topmost nodename to be
passed to qemu. Separate it out into a new function.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

If a mirror job fails to start in -blockdev mode we'd not unplug the
backing files we added first because the code on the error path checked
the wrong value. 'rc' is used as status of the code which added the
images, but the state of the 'block(dev)-mirror' call is stored in 'ret'
at that point.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The virConnectGetDomainCapabilities API accepts either a binary path
to the emulator, or desired guest arch. If guest arch is not given,
then the host arch is assumed.

In the case where the binary is not given, the code tried to find the
emulator binary in the existing list of cached emulator capabilities.
This is not valid since we switched to lazy population of the cache in:

  commit 3dd91af0
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Mon Dec 2 13:04:26 2019 +0000

    qemu: stop creating capabilities at driver startup

As a result of this change, if there are no persistent guests defined
using the requested guest architecture, virConnectGetDomainCapabilities
will fail to find an emulator binary.

The solution is to stop relying on the cached capabilities to find the
binary and instead use the same logic we use to pick default a binary
per arch when populating capabilities.
Tested-by: NBoris Fiuczynski <fiuczy@linux.ibm.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The "ps2" bus is only available on certain machines like x86. On
machines like s390x, we should refuse to add a device to this bus
instead of silently ignoring it.

Looking at the QEMU sources, PS/2 is only available if the QEMU binary
has the "i8042" device, so let's check for that and only allow "ps2"
devices if this QEMU device is available, or if we're on x86 anyway
(so we don't have to fake the QEMU_CAPS_DEVICE_I8042 capability in
all the tests that use <input ... bus='ps2'/> in their xml data).
Reported-by: NSebastian Mitterle <smitterl@redhat.com>
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1763191Signed-off-by: NThomas Huth <thuth@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

LXC driver is not able to retrieve IP addresses from domains. This
function was not implemented yet. It can be done using DHCP lease and
ARP table. Different from QEMU, LXC does not have an agent to fetch
this info, but other sources can be used.
Signed-off-by: NJulio Faracco <jcfaracco@gmail.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

QEMU driver has two functions: qemuGetDHCPInterfaces() and
qemuARPGetInterfaces() that are being used inside only one single
function. They can be turned into generic functions that other drivers
can use. This commit move both from QEMU driver tree to domain conf
tree.
Signed-off-by: NJulio Faracco <jcfaracco@gmail.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Simplify function logic by using g_autofree to free local variables so
that we can remove some goto statements that are used for cleanup.

Introduce a g_autoptr cleanup function for virNodeDeviceDef.
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>

Since commit <60d9ad6f> we require
GnuTLS and since commit <ac0d21c7>
we can actually drop the usage of WITH_GNUTLS.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

If virHostdevManagerGetDefault in qemuhotplugtest fails it works
for quite a while to later segfault when accessing
mgr->activePCIHostdevs.

Report the error details and break on a failed init to see the
real issue right away.
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>
Signed-off-by: NChristian Ehrhardt <christian.ehrhardt@canonical.com>

The libxl driver already tries to call shutdown inhibit callback in the
right places, but only if it's set. That last part was missing,
resulting in premature shutdown when running libvirtd
--timeout=...
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: NJim Fehlig <jfehlig@suse.com>

You normally want to run the locally compiled copy of virsh.  Trying
to run the installed version with the locally compiled library is a
recipe for problems with missing symbols and so on.  By adding tools
to the path we can ensure that (eg) the libguestfs test suite will use
compatible copies of the library and virsh.
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

This has been used in libguestfs and libnbd for quite a while as it
makes the ./run script easier to read and write.

See also:
http://stackoverflow.com/a/9631350Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

This function grabs an agent job but ends a monitor job.
End the agent job instead.

https://bugzilla.redhat.com/show_bug.cgi?id=1792723Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reported-by: NDan Zheng <dzheng@redhat.com>
Fixes: e005c95f

This is only 2 simple typo fixes for wrong documentation wording.
Signed-off-by: NJulio Faracco <jcfaracco@gmail.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Running bootstrap and autoreconf from autogen.sh produced different
files in build-aux directory.  The reason is that gnulib usually have
newer version of these files and overwrites them after the autoreconf
step.

In order to fix it remove the --install and --force options, in addition
introduce --verbose option in order to reflect what bootstrap is doing.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>