https://bugzilla.redhat.com/show_bug.cgi?id=1711789

Starting up or building some types of pools may take a very long
time (e.g. a misconfigured NFS). Holding the pool object locked
throughout the whole time hurts concurrency, e.g. if there's
another thread that is listing all the pools.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

In near future the storage pool object lock will be released
during startPool and buildPool callback (in some backends). But
this means that another thread may acquire the pool object lock
and change its definition rendering the former thread access not
only stale definition but also access freed memory
(virStoragePoolObjAssignDef() will free old def when setting a
new one).

One way out of this would be to have the pool appear as active
because our code deals with obj->def and obj->newdef just fine.
But we can't declare a pool as active if it's not started or
still building up. Therefore, have a boolean flag that is very
similar and forces virStoragePoolObjAssignDef() to store new
definition in obj->newdef even for an inactive pool. In turn, we
have to move the definition to correct place when unsetting the
flag. But that's as easy as calling
virStoragePoolUpdateInactive().

Technically speaking, change made to
storageDriverAutostartCallback() is not needed because until
storage driver is initialized no storage API can run therefore
there can't be anyone wanting to change the pool's definition.
But I'm doing the change there for consistency anyways.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

If there's a persistent storage and user tries to start a new one
with the same name and UUID (e.g. to test new configuration) it
may happen that upon failure we lose the persistent defintion.
Fortunately, we don't remove it from the disk only from the
internal list of the pools.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

There will be more boolean information that we want to pass to
this function. Instead of having them in separate arguments per
each one, use @flags.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

This function is doing much more than plain assigning pool
definition to a pool object. Rename it.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

There is no need for this function to call
virStoragePoolObjEndAPI(). The object is perfectly usable after
return from this function. In fact, all callers will call
virStoragePoolObjEndAPI() eventually.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function comment mistakenly refers to 'poolptr' when in fact
the variable is named 'objptr'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The fact that we're removing a pool object from the list of pools
doesn't mean we want to unlock it. It violates locking policy
too as object locking and unlocking is not done on the same
level.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Store the namespace URI as const char*, instead of in a function.
Suggested-by: NJiri Denemark <jdenemar@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

There is no need to copy and paste the same types pointing
to void all over the place.
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

The virtstoraged daemon will be responsible for providing the storage API
driver functionality. The storage driver is still loaded by the main
libvirtd daemon at this stage, so virtstoraged must not be running at
the same time.
Reviewed-by: NChristophe de Dinechin <dinechin@redhat.com>
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

When running in libvirtd, we are happy for any of the drivers to simply
skip their initialization in virStateInitialize, as other drivers are
still potentially useful.

When running in per-driver daemons though, we want the daemon to abort
startup if the driver cannot initialize itself, as the daemon will be
useless without it.
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

My commit 9b7c4048 was too blind
and my librbd was not new enough to actually compile this part.
Signed-off-by: NJán Tomko <jtomko@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1729292

Fixes: 3aa190f2Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

If there are more than 16 images, the memory allocated in images
might be leaked on subsequent execution(s).
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

When we allow multiple instances of the driver for the same user
account, using a separate root directory, we need to ensure mutual
exclusion. Use a pidfile to guarantee this.

In privileged libvirtd this ends up locking

   /var/run/libvirt/storage/driver.pid

In unprivileged libvirtd this ends up locking

  /run/user/$UID/libvirt/storage/run/driver.pid

NB, the latter can vary depending on $XDG_RUNTIME_DIR
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Add omitted comma for multiple hosts.
Fixes: cdd362e0Signed-off-by: NYi Li <yili@winhong.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NJonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Hosts for rbd are ceph monitor daemons. These have fixed IP addresses,
so they are often referenced by IP rather than hostname for
convenience, or to avoid relying on DNS. Using IPv4 addresses as the
host name works already, but IPv6 addresses require rbd-specific
escaping because the colon is used as an option separator in the
string passed to librados.

Escape these colons, and enclose the IPv6 address in square brackets
so it is distinguished from the port, which is currently mandatory.
Signed-off-by: NYi Li <yili@winhong.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Various binaries are statically linking to libvirt_util.la and
other intermediate libraries we build. These intermediate libs
all get built into the main libvirt.so shared library eventually,
so we can dynamically link to that instead and reduce the on disk
footprint.

In libvirt-daemon RPM:

            virtlockd: 1.6 MB -> 153 KB
             virtlogd: 1.6 MB -> 157 KB
     libvirt_iohelper: 937 KB -> 23 KB

In libvirt-daemon-driver-network RPM:

 libvirt_leaseshelper: 940 KB -> 26 KB

In libvirt-daemon-driver-storage-core RPM:

   libvirt_parthelper: 926 KB -> 21 KB

IOW, about 5.6 MB total space saving in a build done on Fedora 30
x86_64 architecture.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Vim has trouble figuring out the filetype automatically because
the name doesn't follow existing conventions; annotations like
the ones we already have in Makefile.ci help it out.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Firstly, virCommandRun() does report an error on failure (which
in most cases is more accurate than what we overwrite it with).
Secondly, usually errno is not set (or gets overwritten in the
cleanup code) which makes virReportSystemError() report useless
error messages. Drop all virReportSystemError() calls in cases
like this (I've found three occurrences).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

This reverts commit a5e16020.

Getting rid of unistd.h from our headers will require more work than
just fixing the broken mingw build. Revert it until I have a more
complete proposal.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

util/virutil.h bogously included unistd.h. Drop it and replace it by
including it directly where needed.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Add ATTRIBUTE_UNUSED to the volStorageBackendRBDGetFlags stub.

Fixes: 21deeaf0Signed-off-by: NJán Tomko <jtomko@redhat.com>

Refactor code paths which clear strings on cleanup paths to use the
automatic helper.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Use the new refresh volume allocation pool override to skip
computing the actual volume usage if disabled.
Signed-off-by: NJason Dillaman <dillaman@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The librbd API will transparently revert to a slow disk usage
calculation method if the fast-diff map is marked as invalid.
Signed-off-by: NJason Dillaman <dillaman@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The rbd_list method has been deprecated in Ceph >= 14.0.0
in favour of the new rbd_list2 method which populates an
array of structs.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The rbd_list method has a quite unpleasant signature returning an
array of strings in a single buffer instead of an array. It is
being deprecated in favour of rbd_list2. To maintain clarity of
code when supporting both APIs in parallel, split the rbd_list
code out into a separate method.

In splitting this we now honour the rbd_list failures.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

In d16f803d we've tried to solve an issue that after wiping an
image its format might have changed (e.g. from qcow2 to raw) but
libvirt wasn't probing the image format. We fixed this by calling
virStorageBackendRefreshVolTargetUpdate() which is what
refreshPool() would end up calling. But this shortcut is not good
enough because the function is called only for local types of
volumes (like dir, fs, netfs). But now that more backends support
volume wiping we have to call the function with more caution.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

So far we have two branches: either we zero BLOCK_PER_PACKET
(currently 128) block at once, or if we're close to the last block
then we zero out one block at the time. This is very suboptimal.
We know how many block are there left. Might as well just write
them all at once.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

If pool refresh failed, then the internal table of volumes is
probably left in inconsistent or incomplete state anyways. Clear
it out then. This has an advantage that we can move the
virStoragePoolObjClearVols() from those very few backends that
do call it.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

This is a wrapper over refreshPool() call as at all places we are
doing basically the same. Might as well have a single function to
call.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

In bf5cf610 I've fixed a problem where iscsi-direct
backend was reporting only the last LUN. The fix consisted of
moving virStoragePoolObjClearVols() one level up. However, as it
turns out, storage driver already calls it before calling
refreshPool callback (which is
virStorageBackendISCSIDirectRefreshPool() in this case).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

If virStorageBackendISCSIDirectVolWipeZero() fails, it has
already reported an error which is probably specific enough. Do
not overwrite it with some generic one.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

This function reports error for one of the two error paths. This
is unfortunate as a caller see this function failing but doesn't
know right away if an error was reported.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>