https://bugzilla.redhat.com/show_bug.cgi?id=1729292

Fixes: 3aa190f2Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

If there are more than 16 images, the memory allocated in images
might be leaked on subsequent execution(s).
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Add omitted comma for multiple hosts.
Fixes: cdd362e0Signed-off-by: NYi Li <yili@winhong.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Hosts for rbd are ceph monitor daemons. These have fixed IP addresses,
so they are often referenced by IP rather than hostname for
convenience, or to avoid relying on DNS. Using IPv4 addresses as the
host name works already, but IPv6 addresses require rbd-specific
escaping because the colon is used as an option separator in the
string passed to librados.

Escape these colons, and enclose the IPv6 address in square brackets
so it is distinguished from the port, which is currently mandatory.
Signed-off-by: NYi Li <yili@winhong.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Add ATTRIBUTE_UNUSED to the volStorageBackendRBDGetFlags stub.

Fixes: 21deeaf0Signed-off-by: NJán Tomko <jtomko@redhat.com>

Refactor code paths which clear strings on cleanup paths to use the
automatic helper.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Use the new refresh volume allocation pool override to skip
computing the actual volume usage if disabled.
Signed-off-by: NJason Dillaman <dillaman@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The librbd API will transparently revert to a slow disk usage
calculation method if the fast-diff map is marked as invalid.
Signed-off-by: NJason Dillaman <dillaman@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The rbd_list method has been deprecated in Ceph >= 14.0.0
in favour of the new rbd_list2 method which populates an
array of structs.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The rbd_list method has a quite unpleasant signature returning an
array of strings in a single buffer instead of an array. It is
being deprecated in favour of rbd_list2. To maintain clarity of
code when supporting both APIs in parallel, split the rbd_list
code out into a separate method.

In splitting this we now honour the rbd_list failures.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

If pool refresh failed, then the internal table of volumes is
probably left in inconsistent or incomplete state anyways. Clear
it out then. This has an advantage that we can move the
virStoragePoolObjClearVols() from those very few backends that
do call it.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Let's make use of the auto __cleanup capabilities. This also allows
for the cleanup of some goto paths.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Let's make use of the auto __cleanup capabilities cleaning up any
now unnecessary goto paths.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The custom namespaces were originally registered against the storage
pool source struct, but during review this was changed to the top level
storage pool struct. The namespace URIs were not updated to match, so
had a redundant '/source' component.
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Allow for adjustment of RBD configuration options via Storage
Pool XML Namespace adjustments. When namespace arguments are
used to start the pool, add a VIR_WARN to indicate that the
startup was tainted by custom config_opts.

Based off original patch/concept:

https://www.redhat.com/archives/libvir-list/2014-May/msg00940.htmlSigned-off-by: NJohn Ferlan <jferlan@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.

In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.

With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to  find the
author of a particular bit of code.

This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.

The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.
Reviewed-by: NErik Skultety <eskultet@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

As advertised in previous commit, there are three APIs that might
run for quite some time (because they read/write data from/to a
volume) and these three are: downloadVol, uploadVol, wipeVol.
Release pool object lock and reacquire it later to allow more
concurrency.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

In two places the passed pool object argument is marked as
ATTRIBUTE_UNUSED even though it's used right away.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

Now that we can open connections to the secondary drivers on demand,
there is no need to pass a virConnectPtr into all the backend
functions.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Instead of passing around a virConnectPtr object, just open a connection
to the secret driver at time of use. Opening connections on demand will
be beneficial when the secret driver is in a separate daemon. It also
solves the problem that a number of callers just pass in a NULL
connection today which prevents secret lookup working at all.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

In preparation for privatizing the object, use the accessor.

Create/use virStoragePoolObjAddVol in order to add volumes onto list.

Create/use virStoragePoolObjRemoveVol in order to remove volumes from list.

Create/use virStoragePoolObjGetVolumesCount to get count of volumes on list.

For the storage driver, the logic alters when the volumes.obj list grows
to after we've fetched the volobj. This is an optimization of sorts, but
also doesn't "needlessly" grow the volumes.objs list and then just decr
the count if the virGetStorageVol fails.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Use the < 0 rather than == -1 (consistently) for virAsprintf errors.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Add APIs that allow to dynamically register driver backends so that the
list of available drivers does not need to be known during compile time.

This will allow us to modularize the storage driver on runtime.

The file became a garbage dump for all kinds of utility functions over
time. Move them to a separate file so that the files can become a clean
interface for the storage backends.

We had a lot of rados_conf_set and check works.
Use helper virStorageBackendRBDRADOSConfSet for them.
Signed-off-by: NChen Hanxiao <chenhanxiao@gmail.com>

No sense opening a connection only to fail because we don't support the
type of build being attempted.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Rather than use stack allocated state context pointers, let's allocate and
free the state context pointer.  In doing so, we'll shrink the code a bit
since many routines perform the same initialization sequence.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Since none of the callers check the status, let's just alter it to
a static void.

While we're at it - scrap the local runtime variable and just do the
math in the VIR_DEBUG directly.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Remove unnessary translated message marker _()
for the VIR_WARN messages.
Signed-off-by: NChen Hanxiao <chenhanxiao@gmail.com>

s/failed/failed to
Signed-off-by: NChen Hanxiao <chenhanxiao@gmail.com>

Move the enum into a new src/util/virsecret.h, rename it to be
virSecretLookupType. Add a src/util/virsecret.h in order to perform
a couple of simple operations on the secret XML and virSecretLookupTypeDef
for clearing and copying.

This includes quite a bit of collateral damage, but the goal is to remove
the "virStorage*" and replace with the virSecretLookupType so that it's
easier to to add new lookups that aren't necessarily storage pool related.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Rather than inline code secret lookup for rbd/iscsi, use the common function.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Add a new helper that sanitizes error semantics of base64_encode_alloc.

After the recent commits the build didn't work for me. Fix it by
using size_t as the callback argument is using and the correct
formatter. The attempted fixup to use %llu as a formatter was wrong.

%zu is for size_t variables, not uint64 ones. This causes a warning when building on
a 32 bit linux.

%zu is not always synonymous with uint64_t; on 32-bit machines,
size_t is only 32 bits.  Prefer "%lld"/'unsigned long long' when
the variable is under our control, and "%"PRIu64 when we are
stuck with 'uint64_t' from RBD.

Fixes errors such as:

../../src/storage/storage_backend_rbd.c: In function 'virStorageBackendRBDVolWipe':
../../src/storage/storage_backend_rbd.c:1281:15: error: format '%zu' expects argument of type 'size_t', but argument 8 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
     VIR_DEBUG("Need to wipe %zu bytes from RBD image %s/%s",
               ^
../../src/util/virlog.h:90:73: note: in definition of macro 'VIR_DEBUG_INT'
     virLogMessage(src, VIR_LOG_DEBUG, filename, linenr, funcname, NULL, __VA_ARGS__)
                                                                         ^
../../src/storage/storage_backend_rbd.c:1281:5: note: in expansion of macro 'VIR_DEBUG'
     VIR_DEBUG("Need to wipe %zu bytes from RBD image %s/%s",
     ^
Signed-off-by: NEric Blake <eblake@redhat.com>

Checking whether x > 0 before looping over [0..x] items doesn't make
sense and multi-line body must have curly brackets around it.

Best viewed with '-w'.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism
of RBD allows for querying actual volume usage.

Prior to this version there was no easy and fast way to query how
much allocation a RBD volume had inside a Ceph cluster.

To use the fast-diff feature it needs to be enabled per RBD image
and is only supported by Ceph cluster running version Infernalis
(9.2.0) or newer.

Without the fast-diff feature enabled libvirt will report an allocation
identical to the image capacity. This is how libvirt behaves currently.

'virsh vol-info rbd/image2' might output for example:

  Name:           image2
  Type:           network
  Capacity:       1,00 GiB
  Allocation:     124,00 MiB

Newly created volumes will have the fast-diff feature enabled if the
backing Ceph cluster supports it.
Signed-off-by: NWido den Hollander <wido@widodh.nl>