Since version 3.0, qemu has returned disk usage statistics in
guest-get-fsinfo. And since 3.1, it has returned information about the
disk serial number and device node of disks that are targeted by the
filesystem.

Unfortunately, the public API virDomainGetFSInfo() returns the
filesystem info using a virDomainFSInfo struct, and due to API/ABI
guarantees it cannot be extended. So this new information cannot
easily be added to the public API. However, it is possible to add this
new filesystem information to a new virDomainGetGuestInfo() API which
will be based on typed parameters and is thus more extensible.

In order to support these two use cases, I added an internal struct
which the agent code uses to return all of the new data fields. This
internal struct can be converted to the public struct at a cost of some
extra memory allocation.

In a following commit, this additional information will be used within
virDomainGetGuestInfo().
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

This function queries timezone information within the guest and adds
the information to an array of typed parameters with field names
intended to be returned to virDomainGetGuestInfo()
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

This function queries the guest operating system information and adds
the returned information to an array of typed parameters with field
names intended to be returned in virDomainGetGuestInfo().
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

This function fetches the list of logged-in users from the qemu agent
and adds them to a list of typed parameters so that they can be used
internally in libvirt.

Also add some basic tests for the function.
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Add daemon and client code to serialize/deserialize
virDomainGetGuestInfo().
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

This API is intended to aggregate several guest agent information
queries and is ispired by stats API virDomainListGetStats(). It is
anticipated that this information will be provided by a guest agent
running within the domain.
Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the internals into qemuDomainSnapshotDiskDataCollectOne to make it
obvious what's happening after moving more code here.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Soon we'll allow more protocols and storage types with snapshots where
we in some cases can't check whether the storage already exists.
Restrict the sanity checks whether the destination images exist or not
for local storage where it's easy. For any other case we will fail
later.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Refactor the code to avoid having a cleanup label. This will simplify
the change necessary when restricting this check in an upcoming patch.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

dd->src is always allocated in this function as it contains the new
source for the snapshot which is meant to replace the disk source.

The label handling code executed if that source was not present thus is
dead code. Remove it.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

While the VM is running the persistent source of a disk might differ
e.g. as the 'newDef' was redefined. Our snapshot code would blindly
rewrite the source of such disk if it shared the 'target'. Fix this by
checking whether the source is the same in the first place.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Using inline authentication for storage volumes will not work properly
as libvirt requires use of the secret driver for the auth data and
thus would not be able to represent the passwords stored in the backing
store string.

Make sure that the backing store parsers return 1 which is a sign for
the caller to not use the file in certain cases.

The test data include iscsi via a json pseudo-protocol string and URIs
with the userinfo part being present.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virStorageFileGetMetadataRecurse would include files in the backing
chain which would not really be usable by libvirt directly e.g.
when such file would be promoted to the top layer by an active block
commit as for example inline authentication data can't be represented in
the VM xml file. The idea is to use secrets for this.

With the changes to the backing store string parsers we can report and
propagate if such a thing is present in the configuration and thus start
skipping those files in the backing chain traversal code. This approach
still allows to report the appropriate backing store string in the
storage driver which doesn't directly use the backing file.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virStorageFileGetMetadata does not report error if we can't interrogate
the file somehow. Clarify this in the description of the @report_broken
flag as it implies we should report an error in that case. The problem
is that we don't know whether there's a problem and unfortunately just
offload it to qemu.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Introduce new semantics to virStorageSourceNewFromBacking and some
of the helpers used by it which propagate the return value from the
callers.

The new return value introduced by this patch allows to notify the
calller that the parsed virStorageSource correctly describes the source
but contains data such as inline authentication which libvirt does not
want to support directly. This means that such file would e.g. unusable
as a storage source (e.g. when actively commiting the overlay to it) or
would not work with blockdev.

The caller will then be able to decide whether to consider this backing
file as viable or just fall back to qemu dealing with it.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Modify testBackingParse to allow testing other return values of the
backing store string parser.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Return the parsed storage source via an pointer in arguments and return
an integer from the function. Describe the semantics with a comment for
the function and adjust callers to the new semantics.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virStorageSourceParseBackingURI will report special return values in
some cases. Preserve it in virStorageSourceParseBackingJSONUriStr.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Return the storage source definition via a pointer in the arguments and
document the returned values. This will simplify the possibility to
ignore certain backing store types which are not representable by
libvirt.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Automatically clean the temporary buffer and get rid of the cleanup
label.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

While it's a bad idea to use userinfo to pass credentials via a URI add
a test that we at least do the correct thing.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Automatically free the 'root' temporary variable to get rid of some
complexity.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Automatically clean the 'uri' variable and get rid of the 'cleanup'
label.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

There is no cleanup code.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Automatically free the intermediate JSON data to get rid of the cleanup
section.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Commit fb64e176 forgot to delete the check that short-circuits the
disk alias creation if the alias is already present. The side effect
of this is that the creation qomName which is necessary to be able to
refer to disk frontends when -blockdev is used was skipped when user
aliases are used.

Fix it by deleting the check. Also prevent any potential memory leaks
from calling this function repeatedly by creating the qomName only when
it's not present.

https://bugzilla.redhat.com/show_bug.cgi?id=1741838Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Currently, whenever there's a regular EOF on the console stream
or an error the virStreamAbort() is called regardless. While this
may not actually break anything, we should call virStreamFinish()
to let the daemon know we've successfully received all the data
and are shutting down the stream gracefully.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NRoman Bolshakov <r.bolshakov@yadro.com>

In my recent patches I've introduced
virStoragePoolObjIsStarting() which is then used to protect
storage pool definition when the pool object is locked and
unlocked during long running jobs. Well, my patches did not
anticipate that @obj can be NULL under 'cleanup' label in
storagePoolCreateXML() (for instance when parsing XML fails).
This imperfection is causing libvirtd to crash then.

Fixes: 13284a6b storage_driver: Protect pool def during startup and build
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NMartin Kletzander <mkletzan@redhat.com>

Regular VM shutdown triggers the error for existing session of virsh
console and it returns with non-zero exit code:
  error: internal error: console stream EOF

The message and status code are misleading because there's no real
error. virStreamRecv returns 0 correctly when EOF is reached.

Existing implementations of esx, fd, and remote streams behave the same
for virStreamFinish and virStreamAbort: they close the stream. So, we
can continue to use virStreamAbort to handle EOF and errors from
virStreamRecv but additonally we can report error if virStreamAbort
fails.

Fixes: 29f2b524 ("tools: console: pass stream/fd errors to user")
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

A library has to be built with -flat_namespace to get all references to
global symbols indirected. That can also be achieved with two-level
namespace interposition but we're not using explicit symbol
interposition since it's more verbose and requires massive changes to
the mocks.

This provides a way to interpose a mock for virQEMUCapsProbeHostCPU from
qemucpumock and fixes domaincapstest on macOS.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

gnulib headers change stat, lstat and open to replacement functions,
even for function definitions. This effectively disables standard
library overrides in virfilewrapper and virmockstathelpers since they
are never reached.

Rename the functions and provide a declartion that uses correct
assembler name for the mocks.

This fixes firmware lookup in domaincapstest on macOS.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

Test executables and mocks have assumption that any symbol can be
replaced with LD_PRELOAD. That's not a case for macOS unless flat
namespace is used, because every external symbol reference records the
library to be looked up. And the symbols cannot be replaced unless dyld
interposing is used.

Setting DYLD_FORCE_FLAT_NAMESPACE changes symbol lookup behaviour to be
similar to Linux dynamic linker. It's more lightweight solution than
explicitly decorating all mock symbols as interpositions and building
libvirt as interposable dynamic library.

This fixes vircryptotest and allows to proceed other tests that rely on
mocks a little bit further.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

macOS syscall interface (/usr/lib/system/libsystem_kernel.dylib) has
three kinds of stat but only one of them can be used to fill
"struct stat": stat$INODE64.

virmockstathelpers looks up regular stat instead of stat$INODE64.  That
causes a failure in qemufirmwaretest because "struct stat" is laid out
differently from the values returned by stat.

Introduce VIR_MOCK_REAL_INIT_ALIASED that can be used to lookup
stat$INODE64 and lstat$INODE64 and use it to setup real functions on
macOS.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

>From ld(1):

  By default all references resolved to a dynamic library record the
  library to which they were resolved. At runtime, dyld uses that
  information to directly resolve symbols. The alternative is to use the
  -flat_namespace option.  With flat namespace, the library is not
  recorded.  At runtime, dyld will search each dynamic library in load
  order when resolving symbols. This is slower, but more like how other
  operating systems resolve symbols.

That fixes the set of tests that preload a mock library to replace
library symbols:
  qemublocktest
  qemumonitorjsontest
  viriscsitest
  virmacmaptest
  virnetserverclienttest
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

/tmp is a symbolic link to /private/tmp on macOS. That causes failures
in commandtest, because getcwd returns /private/tmp and the expected
output doesn't match to "CWD: /tmp".

Rathern than making a copy of commanddata solely for macOS, the /private
prefix is stripped.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

macOS has two kinds of loadable libraries: MH_BUNDLE, and MH_DYLIB.
bundle is used for plugins that are loaded with dlopen/dlsym/dlclose.
And there's no way to preload a bundle into an application. dynamic
linker (dyld) will reject it when finds it in DYLD_INSERT_LIBRARIES.

Unfortunately, a bundle is built if -module flag is provided to libtool.
The flag has been removed to build dylibs with ".dylib" suffix.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

In preparation libtool "-module" flag removal, add lib prefix to all
mock shared objects.

While at it, introduce VIR_TEST_MOCK macros that makes path out of mock
name to be used with VIR_TEST_PRELOAD or VIR_TEST_MAIN_PRELOAD.  That,
hopefully, improves readability, reduces line length and allows to
tailor VIR_TEST_MOCK for specific platform if it has shared library
suffix different from ".so".
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

LD_PRELOAD has no effect on macOS. Instead, dyld(1) provides a way for
symbol hooking via DYLD_INSERT_LIBRARIES. The variable should contain
colon-separated paths to the dylibs to be inserted.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

getnameinfo on macOS formats certain IPv6 addresses as IPv4-translated
addresses. The following pattern has been observed:
  ::ffff is formated as ::0.0.255.255
  ::fffe is formated as ::0.0.255.254
  ::ffff:0 is formated as ::255.255.0.0
  ::fffe:0 is formated as ::255.254.0.0
  ::ffff:0:0 is formated as ::ffff:0.0.0.0
  ::fffe:0:0 is formated as ::fffe:0:0
  ::ffff:0:0:0 is formated as ::ffff:0:0:0

The getnameinfo behavior causes a failure for:
  DO_TEST_PARSE_AND_FORMAT("::ffff", AF_UNSPEC, true);

Use non-ambigious IPv6 for parse/format testing.
Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>

getaddrinfo on macOS doesn't interpret octal IPv4 addresses. Only
inet_aton can be used for that. Therefore, from macOS standpoint
"0177.0.0.01" is not the same as "127.0.0.1".

The issue was also discovered by python and dotnet core:
  https://bugs.python.org/issue27612
  https://github.com/dotnet/corefx/issues/8362Signed-off-by: NRoman Bolshakov <r.bolshakov@yadro.com>