From commit id 'd53bbfd1'

Found one core and one possible memory leak. Core seen during local
virt-test/tp_libvirt run for the vol_create_from test. The memory leak
was seen by inspection during a review of all VIR_APPEND_ELEMENT changes

In storage_backend_disk/virStorageBackendDiskMakeDataVol(), the 'vol'
needs to be kept around since it's used later, so use the _COPY macro.
This caused a segv in libvirtd:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe87c3700 (LWP 6919)]
virStorageBackendDiskMakeDataVol (vol=0x0, groups=0x7fffc8000d70, pool=0x7fffc8002460) at storage/storage_backend_disk.c:66
66          if (vol->target.path == NULL) {

In storage_backend_rbd/virStorageBackendRBDRefreshPool() there's a failure
path where the 'vol' needs to go through virStorageVolDefFree() since it
wouldn't be appended.

Some of these are leftovers from renaming the files, others
are just typos.

Also introduce an ugly awk script to enforce this.

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In storageVolLookupByPath the provided path is "sanitized" at first.
This removes some extra slashes and stuff. When the lookup of the volume
fails the original path is used which makes it hard to trace errors in
some cases.

Improve the error message to print the sanitized path along with the
user provided path if they are not equal.

When looking up a volume by path on a non-local filesystem don't use the
"cleaned" path that might be mangled in such a way that it will differ
from a path provided by a storage backend.

Skip the cleanup step for gluster, sheepdog and RBD.

Pools that are not backed by files in the filesystem cause problems with
some APIs. Error out when attempting to upload a volume in such a pool
as currently we expect a local file representation for it.

Auditing all callers of virCommandRun and virCommandWait that
passed a non-NULL pointer for exit status turned up some
interesting observations.  Many callers were merely passing
a pointer to avoid the overall command dying, but without
caring what the exit status was - but these callers would
be better off treating a child death by signal as an abnormal
exit.  Other callers were actually acting on the status, but
not all of them remembered to filter by WIFEXITED and convert
with WEXITSTATUS; depending on the platform, this can result
in a status being reported as 256 times too big.  And among
those that correctly parse the output, it gets rather verbose.
Finally, there were the callers that explicitly checked that
the status was 0, and gave their own message, but with fewer
details than what virCommand gives for free.

So the best idea is to move the complexity out of callers and
into virCommand - by default, we return the actual exit status
already cleaned through WEXITSTATUS and treat signals as a
failed command; but the few callers that care can ask for raw
status and act on it themselves.

* src/util/vircommand.h (virCommandRawStatus): New prototype.
* src/libvirt_private.syms (util/command.h): Export it.
* docs/internals/command.html.in: Document it.
* src/util/vircommand.c (virCommandRawStatus): New function.
(virCommandWait): Adjust semantics.
* tests/commandtest.c (test1): Test it.
* daemon/remote.c (remoteDispatchAuthPolkit): Adjust callers.
* src/access/viraccessdriverpolkit.c (virAccessDriverPolkitCheck):
Likewise.
* src/fdstream.c (virFDStreamCloseInt): Likewise.
* src/lxc/lxc_process.c (virLXCProcessStart): Likewise.
* src/qemu/qemu_command.c (qemuCreateInBridgePortWithHelper):
Likewise.
* src/xen/xen_driver.c (xenUnifiedXendProbe): Simplify.
* tests/reconnect.c (mymain): Likewise.
* tests/statstest.c (mymain): Likewise.
* src/bhyve/bhyve_process.c (virBhyveProcessStart)
(virBhyveProcessStop): Don't overwrite virCommand error.
* src/libvirt.c (virConnectAuthGainPolkit): Likewise.
* src/openvz/openvz_driver.c (openvzDomainGetBarrierLimit)
(openvzDomainSetBarrierLimit): Likewise.
* src/util/virebtables.c (virEbTablesOnceInit): Likewise.
* src/util/viriptables.c (virIpTablesOnceInit): Likewise.
* src/util/virnetdevveth.c (virNetDevVethCreate): Fix debug
message.
* src/qemu/qemu_capabilities.c (virQEMUCapsInitQMP): Add comment.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSINodeUpdate): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

These timeout values make librados/librbd return -ETIMEDOUT when a
operation is blocking due to a failing/unreachable Ceph cluster.

By having the operations time out libvirt will not block.

With this information it's easier for the user to debug what is
going wrong.

There has been a new field introduced in iscsiadm --mode session
output [1], but our regex only expects four fields. This breaks
startup of iscsi pools:
error: Failed to start pool iscsi
error: internal error: cannot find session

Fix this by ignoring anything after the fourth field.

https://bugzilla.redhat.com/show_bug.cgi?id=1067173

[1] https://github.com/mikechristie/open-iscsi/commit/181af9a

The internal pools were an idea in one of the first iterations of the
gluster series, which we decided not to use. Somehow the patch still
got pushed. Remove it as the internal flag isn't needed.

This reverts commit 362da820.

In a44b7b87 I've introduced a function
that initializes a storage file wrapper object on gluster based volumes.

The initialization function leaks the private data pointer in case of
failure. This patch fixes it.

Reported by John Ferlan.

In commit e3226818 I accidentally added
twice a typedef for virStorageFileBackend when I moved it between files
across patch iterations. The double declaration breaks build on older
compilers in RHEL5 and FreeBSD.

Remove the spurious definition.

Implement storage backend functions to deal with gluster volumes and
implement the "stat" and "unlink" backend APIs.

Implement the "stat" and "unlink" function for "file" volumes and "stat"
for "block" volumes using the regular system calls.

Add APIs that will allow to use the storage driver to assist in
operations on files even for remote filesystems without native
representation as files in the host.

virGetStorageVol can return NULL on out-of-memory. If it does, cleanly
abort the volume clone operation.
Signed-off-by: NMichael Chapman <mike@very.puzzling.org>

This reverts commit 67ccf91b.
We only generate the volume key after we've built it, but the storage
driver expects it to be filled after createVol finishes.
Squash the volume building back with creating to fulfill this
expectation.

This reverts commit af1fb38f.
With it, creating new logical volumes fails:
https://www.redhat.com/archives/libvir-list/2014-February/msg00658.html

In the storage driver, we expect CreateVol to fill out the volume key,
but the LVM backend fills the key with the uuid reported by lvs after the
logical volume is created.

Two unused global variables, and DBUS_TYPE_INVALID used as a const
char*.
Signed-off-by: NEric Blake <eblake@redhat.com>

Extract the metadata setting code into a separate function for future
use.

This new RBD format supports snapshotting and cloning. By having
libvirt create images in format 2 end-users of the created images
can benefit from the new RBD format.

Older versions of libvirt can work with this new RBD format as long
as librbd supports format 2. RBD format is supported by librbd since
version 0.56 (Ceph Bobtail).
Signed-off-by: NWido den Hollander <wido@widodh.nl>

When restarting sheepdog pool, all volumes are missing.
This patch add automatically all volume from the added pool.

Adding last Daniel P. Berrange's syntaxes correction.
Adding vol on separeted function 'inspired' from parallels_storage :
parallelsAddDiskVolume

The return value of virGetFCHostNameByWWN is a strdup'ed string.
Also add comments to declare that the caller should take care of
freeing it.

The "checkPool" is a bit different for pool with "fc_host"
type source adapter, since the vHBA it's based on might be
not created yet (it's created by "startPool", which is
involked after "checkPool" in storageDriverAutostart). So it
should not fail, otherwise the "autostart" of the pool will
fail either.

The problem is easy to reproduce:
    * Enable "autostart" for the pool
    * Restart libvirtd service
    * Check the pool's state

For pool which relies on remote resources, such as a "iscsi" type
pool, since how long it takes to export the corresponding devices
to host's sysfs is really depended, it could depend on the network
connection, it also could depend on the host's udev procedures. So
it's likely that the volumes are not able to be detected during pool
starting process, polling the sysfs doesn't work, since we don't
know how much time is best for the polling, and even worse, the
volumes could still be not detected or partly not detected even after
the polling.  So we end up with a documentation to prompt the fact,
in virsh manual.

And as a small improvement, let's explicitly say no LUNs found in
the debug log in that case.

The public virConnectRef and virConnectClose API are just thin
wrappers around virObjectRef/virObjectRef, with added object
validation and an error reset.  Within our backend drivers, use
of the object validation is just an inefficiency since we always
pass valid objects.  More important to think about is what
happens with the error reset; our uses of virConnectRef happened
to be safe (since we hadn't encountered any earlier errors), but
in several cases the use of virConnectClose could lose a real
error.

Ideally, we should also avoid calling virConnectOpen() from
within backend drivers - but that is a known situation that
needs much more design work.

* src/qemu/qemu_process.c (qemuProcessReconnectHelper)
(qemuProcessReconnect): Avoid nested public API call.
* src/qemu/qemu_driver.c (qemuAutostartDomains)
(qemuStateInitialize, qemuStateStop): Likewise.
* src/qemu/qemu_migration.c (doPeer2PeerMigrate): Likewise.
* src/storage/storage_driver.c (storageDriverAutostart):
Likewise.
* src/uml/uml_driver.c (umlAutostartConfigs): Likewise.
* src/lxc/lxc_process.c (virLXCProcessAutostartAll): Likewise.
(virLXCProcessReboot): Likewise, and avoid leaking conn on error.
Signed-off-by: NEric Blake <eblake@redhat.com>

To allow using the storage driver APIs to do operation on generic domain
disks we will need to introduce internal storage pools that will give is
a base to support this stuff even on files that weren't originally
defined as a part of the pool.

This patch introduces the 'internal' flag for a storage pool that will
prevent it from being listed along with the user defined storage pools.

Separate the steps to create libvirt's volume metadata from the actual
volume building process.

Separate the steps to create libvirt's volume metadata from the actual
volume building process.

Separate the steps to create libvirt's volume metadata from the actual
volume building process.

Separate the steps to create libvirt's volume metadata from the actual
volume building process. This is already done for regular file based
pools to allow job support for storage APIs.

Implement the "deleteVol" storage backend function for gluster volumes.

Include the name of the storage backend in the error message instead of
just the number.

Change code ordering to avoid the need for a forward declaration.

The comment was talking about creating the pool while the function is
deleting it. Fix the mismatch.

The commit cad3cf9a introduced a crash
due to wrong order of parameters being passed to the function.  When
deleting an element, the function decreased the iterator instead of
count and if listing volumes after that (or undefining the pool, NULL
was being dereferenced.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>