Noticed during my work on storage struct cleanups.

* src/storage/storage_backend_disk.c
(virStorageBackendDiskPartBoundaries): Fix spelling errors.
Signed-off-by: NEric Blake <eblake@redhat.com>

Now that we have a common struct, it's time to start using it!
Since external snapshots make a longer backing chain, it is
only natural to use the same struct for the file created by
the snapshot as what we use for <domain> disks.

* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Use common
struct instead of open-coded duplicate fields.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefClear)
(virDomainSnapshotDiskDefParseXML, virDomainSnapshotAlignDisks)
(virDomainSnapshotDiskDefFormat)
(virDomainSnapshotDiskGetActualType): Adjust clients.
* src/qemu/qemu_conf.c (qemuTranslateSnapshotDiskSourcePool):
Likewise.
* src/qemu/qemu_driver.c (qemuDomainSnapshotDiskGetSourceString)
(qemuDomainSnapshotCreateInactiveExternal)
(qemuDomainSnapshotPrepareDiskExternalOverlayActive)
(qemuDomainSnapshotPrepareDiskExternal)
(qemuDomainSnapshotPrepare)
(qemuDomainSnapshotCreateSingleDiskActive): Likewise.
* src/storage/storage_driver.c
(virStorageFileInitFromSnapshotDef): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Another enum moved to util/, this time the fallout from renaming
is not quite as large.

* src/conf/domain_conf.h (virDomainDiskProtocol): Move...
* src/util/virstoragefile.h (virStorageNetProtocol): ...and
rename.
* src/conf/domain_conf.c: Update clients.
* src/qemu/qemu_command.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/qemu/qemu_migration.c: Likewise.
* src/storage/storage_backend.c: Likewise.
* src/storage/storage_backend_gluster.c: Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

A continuation of the migration of disk details to virstoragefile.
This patch moves a single enum, but converting the name has quite
a bit of fallout.

* src/conf/domain_conf.h (virDomainDiskType): Move...
* src/util/virstoragefile.h (virStorageType): ...and rename.
* src/bhyve/bhyve_command.c (bhyveBuildDiskArgStr)
(virBhyveProcessBuildLoadCmd): Update clients.
* src/conf/domain_conf.c (virDomainDiskSourceDefParse)
(virDomainDiskDefParseXML, virDomainDiskSourceDefFormatInternal)
(virDomainDiskDefFormat, virDomainDiskGetActualType)
(virDomainDiskDefForeachPath, virDomainDiskSourceIsBlockType):
Likewise.
* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefParseXML)
(virDomainSnapshotAlignDisks, virDomainSnapshotDiskDefFormat):
Likewise.
* src/esx/esx_driver.c (esxAutodetectSCSIControllerModel)
(esxDomainDefineXML): Likewise.
* src/locking/domain_lock.c (virDomainLockManagerAddDisk):
Likewise.
* src/lxc/lxc_controller.c
(virLXCControllerSetupLoopDeviceDisk)
(virLXCControllerSetupNBDDeviceDisk)
(virLXCControllerSetupLoopDevices, virLXCControllerSetupDisk):
Likewise.
* src/parallels/parallels_driver.c (parallelsGetHddInfo):
Likewise.
* src/phyp/phyp_driver.c (phypDiskType): Likewise.
* src/qemu/qemu_command.c (qemuGetDriveSourceString)
(qemuDomainDiskGetSourceString, qemuBuildDriveStr)
(qemuBuildCommandLine, qemuParseCommandLineDisk)
(qemuParseCommandLine): Likewise.
* src/qemu/qemu_conf.c (qemuCheckSharedDevice)
(qemuTranslateDiskSourcePool)
(qemuTranslateSnapshotDiskSourcePool): Likewise.
* src/qemu/qemu_domain.c (qemuDomainDeviceDefPostParse)
(qemuDomainDetermineDiskChain): Likewise.
* src/qemu/qemu_driver.c (qemuDomainGetBlockInfo)
(qemuDomainSnapshotPrepareDiskExternalBackingInactive)
(qemuDomainSnapshotPrepareDiskExternalBackingActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayInactive)
(qemuDomainSnapshotPrepareDiskInternal)
(qemuDomainSnapshotPrepare)
(qemuDomainSnapshotCreateSingleDiskActive): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsSafe): Likewise.
* src/security/security_apparmor.c
(AppArmorRestoreSecurityImageLabel)
(AppArmorSetSecurityImageLabel): Likewise.
* src/security/security_dac.c (virSecurityDACSetSecurityImageLabel)
(virSecurityDACRestoreSecurityImageLabelInt)
(virSecurityDACSetSecurityAllLabel): Likewise.
* src/security/security_selinux.c
(virSecuritySELinuxRestoreSecurityImageLabelInt)
(virSecuritySELinuxSetSecurityImageLabel)
(virSecuritySELinuxSetSecurityAllLabel): Likewise.
* src/storage/storage_backend.c (virStorageFileBackendForType):
Likewise.
* src/storage/storage_backend_fs.c (virStorageFileBackendFile)
(virStorageFileBackendBlock): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageFileBackendGluster): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc, vboxAttachDrives)
(vboxDomainAttachDeviceImpl, vboxDomainDetachDevice): Likewise.
* src/vmware/vmware_conf.c (vmwareVmxPath): Likewise.
* src/vmx/vmx.c (virVMXParseDisk, virVMXFormatDisk)
(virVMXFormatFloppy): Likewise.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenParseSxpr)
(xenFormatSxprDisk): Likewise.
* src/xenxs/xen_xm.c (xenParseXM, xenFormatXMDisk): Likewise.
* tests/securityselinuxlabeltest.c (testSELinuxLoadDef):
Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Continuing the refactoring of host-side storage descriptions out
of conf/domain_conf and into util/virstoragefile, this patch
focuses on details about a host name/port/transport as used by
a network storage volume.

* src/conf/domain_conf.h (virDomainDiskProtocolTransport)
(virDomainDiskHostDef, virDomainDiskHostDefClear)
(virDomainDiskHostDefFree, virDomainDiskHostDefCopy): Move...
* src/util/virstoragefile.h (virStorageNetHostTransport)
(virStorageNetHostDef, virStorageNetHostDefClear)
(virStorageNetHostDefFree, virStorageNetHostDefCopy): ...here,
with better names.
* src/util/virstoragefile.c (virStorageNetHostDefClear)
(virStorageNetHostDefFree, virStorageNetHostDefCopy): Moved from...
* src/conf/domain_conf.c (virDomainDiskHostDefClear)
(virDomainDiskHostDefFree, virDomainDiskHostDefCopy): ...here.
(virDomainDiskSourceDefClear, virDomainDiskSourceDefParse)
(virDomainDiskSourceDefFormatInternal): Adjust callers.
* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefClear):
Likewise.
* src/qemu/qemu_command.c (qemuAddRBDHost)
(qemuParseDriveURIString, qemuParseNBDString)
(qemuBuildNetworkDriveURI, qemuParseCommandLineDisk)
(qemuParseCommandLine, qemuGetDriveSourceString): Likewise.
* src/qemu/qemu_command.h: Likewise.
* src/qemu/qemu_conf.c (qemuAddISCSIPoolSourceHost)
(qemuTranslateDiskSourcePool): Likewise.
* src/qemu/qemu_driver.c
(qemuDomainSnapshotCreateSingleDiskActive)
(qemuDomainSnapshotUndoSingleDiskActive): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageFileBackendGlusterInit): Likewise.
* src/storage/storage_driver.c (virStorageFileFree)
(virStorageFileInitInternal): Likewise.
* src/storage/storage_driver.h (_virStorageFile): Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Use the previously implemented function to lookup glusterfs source pools
for the netfs pool to lookup native gluster pools too.

https://bugzilla.redhat.com/show_bug.cgi?id=1072714

Use the "gluster" command line tool to retrieve information about remote
volumes on a gluster server to allow storage pool source lookup.

Unfortunately gluster doesn't provide a management library so that we
could use that directly, instead the RPC calls are hardcoded in the
command line tool.

Extract the NFS related stuff into a separate function and tidy up the
rest of the code so we can reuse it to add gluster backend detection.

Additionally avoid reporting of errors from "showmount" and return an
empty source list instead. This will help when adding other detection
backends.

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

According to our documentation the "key" value has the following
meaning: "Providing an identifier for the volume which identifies a
single volume." The currently used keys for gluster volumes consist of
the gluster volume name and file path. This can't be considered unique
as a different storage server can serve a volume with the same name.

Unfortunately I wasn't able to figure out a way to retrieve the gluster
volume UUID which would avoid the possibility of having two distinct
keys identifying a single volume.

Use the full URI as the key for the volume to avoid the more critical
ambiguity problem and document the possible change to UUID.

Remove the original VolOpen implementation, which is now only used in
one spot.

And drop the original UpdateVolInfo. Makes it a bit easier to follow
the function usage.

And change the int parameter to an explicit bool.

The libgfapi function glfs_fini doesn't tolerate NULL pointers. Add a
check on the error paths as it's possible to crash libvirtd if the
gluster volume can't be initialized.

It's finally time to start tracking disk backing chains in
<domain> XML.  The first step is to start refactoring code
so that we have an object more convenient for representing
each host source resource in the context of a single guest
<disk>.  Ultimately, I plan to move the new type into src/util
where it can be reused by virStorageFile, but to make the
transition easier to review, this patch just creates the
new type then fixes everything until it compiles again.

* src/conf/domain_conf.h (_virDomainDiskDef): Split...
(_virDomainDiskSourceDef): ...to new struct.
(virDomainDiskAuthClear): Use new type.
* src/conf/domain_conf.c (virDomainDiskDefFree): Split...
(virDomainDiskSourceDefClear): ...to new function.
(virDomainDiskGetType, virDomainDiskSetType)
(virDomainDiskGetSource, virDomainDiskSetSource)
(virDomainDiskGetDriver, virDomainDiskSetDriver)
(virDomainDiskGetFormat, virDomainDiskSetFormat)
(virDomainDiskAuthClear, virDomainDiskGetActualType)
(virDomainDiskDefParseXML, virDomainDiskSourceDefFormat)
(virDomainDiskDefFormat, virDomainDiskDefForeachPath)
(virDomainDiskDefGetSecurityLabelDef)
(virDomainDiskSourceIsBlockType): Adjust all users.
* src/lxc/lxc_controller.c (virLXCControllerSetupDisk):
Likewise.
* src/lxc/lxc_driver.c (lxcDomainAttachDeviceMknodHelper):
Likewise.
* src/qemu/qemu_command.c (qemuAddRBDHost, qemuParseRBDString)
(qemuParseDriveURIString, qemuParseGlusterString)
(qemuParseISCSIString, qemuParseNBDString)
(qemuDomainDiskGetSourceString, qemuBuildDriveStr)
(qemuBuildCommandLine, qemuParseCommandLineDisk)
(qemuParseCommandLine): Likewise.
* src/qemu/qemu_conf.c (qemuCheckSharedDevice)
(qemuAddISCSIPoolSourceHost, qemuTranslateDiskSourcePool):
Likewise.
* src/qemu/qemu_driver.c (qemuDomainUpdateDeviceConfig)
(qemuDomainPrepareDiskChainElement)
(qemuDomainSnapshotCreateInactiveExternal)
(qemuDomainSnapshotPrepareDiskExternalBackingInactive)
(qemuDomainSnapshotPrepareDiskInternal)
(qemuDomainSnapshotPrepare)
(qemuDomainSnapshotCreateSingleDiskActive)
(qemuDomainSnapshotUndoSingleDiskActive)
(qemuDomainBlockPivot, qemuDomainBlockJobImpl)
(qemuDomainBlockCopy, qemuDomainBlockCommit): Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsSafe): Likewise.
* src/qemu/qemu_process.c (qemuProcessGetVolumeQcowPassphrase)
(qemuProcessInitPasswords): Likewise.
* src/security/security_selinux.c
(virSecuritySELinuxSetSecurityFileLabel): Likewise.
* src/storage/storage_driver.c (virStorageFileInitFromDiskDef):
Likewise.
* tests/securityselinuxlabeltest.c (testSELinuxLoadDef):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

If we cannot stat/open a file on pool refresh, returning -1 aborts
the refresh and the pool is undefined.

Only treat missing files as fatal unless VolOpenCheckMode is called
with the VIR_STORAGE_VOL_OPEN_ERROR flag. If this flag is missing
(when it's called from virStorageBackendProbeTarget in
virStorageBackendFileSystemRefresh), only emit a warning and return
-2 to let the caller skip over the file.

https://bugzilla.redhat.com/show_bug.cgi?id=977706

Without this, using /dev/mapper as a directory pool
fails in virStorageBackendUpdateVolTargetInfoFD:
cannot seek to end of file '/dev/mapper/control': Illegal seek

Skip over character devices by default.

https://bugzilla.redhat.com/show_bug.cgi?id=710866

Remove the 'StorageBackend' from names of the functions and fix
indentation.

virStorageBackendISCSISession only needs the path of the source
device and virStorageBackendISCSIRescanLUNs doesn't need the pool
at all.

This will allow the functions to be moved to src/util.

Create ISCSIConnection{Login,Logout} wrappers for that.

The only storage-specific parameter is the pool object, which
is only used for passing to the callback function.

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Coverity found an issue in lxc_driver and uml_driver that we don't
check the return value of register functions.

I've also updated all other places and unify the way we check the
return value.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

It quacks like a bool.

From commit id 'd53bbfd1'

Found one core and one possible memory leak. Core seen during local
virt-test/tp_libvirt run for the vol_create_from test. The memory leak
was seen by inspection during a review of all VIR_APPEND_ELEMENT changes

In storage_backend_disk/virStorageBackendDiskMakeDataVol(), the 'vol'
needs to be kept around since it's used later, so use the _COPY macro.
This caused a segv in libvirtd:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe87c3700 (LWP 6919)]
virStorageBackendDiskMakeDataVol (vol=0x0, groups=0x7fffc8000d70, pool=0x7fffc8002460) at storage/storage_backend_disk.c:66
66          if (vol->target.path == NULL) {

In storage_backend_rbd/virStorageBackendRBDRefreshPool() there's a failure
path where the 'vol' needs to go through virStorageVolDefFree() since it
wouldn't be appended.

Some of these are leftovers from renaming the files, others
are just typos.

Also introduce an ugly awk script to enforce this.

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In storageVolLookupByPath the provided path is "sanitized" at first.
This removes some extra slashes and stuff. When the lookup of the volume
fails the original path is used which makes it hard to trace errors in
some cases.

Improve the error message to print the sanitized path along with the
user provided path if they are not equal.

When looking up a volume by path on a non-local filesystem don't use the
"cleaned" path that might be mangled in such a way that it will differ
from a path provided by a storage backend.

Skip the cleanup step for gluster, sheepdog and RBD.

Pools that are not backed by files in the filesystem cause problems with
some APIs. Error out when attempting to upload a volume in such a pool
as currently we expect a local file representation for it.

Auditing all callers of virCommandRun and virCommandWait that
passed a non-NULL pointer for exit status turned up some
interesting observations.  Many callers were merely passing
a pointer to avoid the overall command dying, but without
caring what the exit status was - but these callers would
be better off treating a child death by signal as an abnormal
exit.  Other callers were actually acting on the status, but
not all of them remembered to filter by WIFEXITED and convert
with WEXITSTATUS; depending on the platform, this can result
in a status being reported as 256 times too big.  And among
those that correctly parse the output, it gets rather verbose.
Finally, there were the callers that explicitly checked that
the status was 0, and gave their own message, but with fewer
details than what virCommand gives for free.

So the best idea is to move the complexity out of callers and
into virCommand - by default, we return the actual exit status
already cleaned through WEXITSTATUS and treat signals as a
failed command; but the few callers that care can ask for raw
status and act on it themselves.

* src/util/vircommand.h (virCommandRawStatus): New prototype.
* src/libvirt_private.syms (util/command.h): Export it.
* docs/internals/command.html.in: Document it.
* src/util/vircommand.c (virCommandRawStatus): New function.
(virCommandWait): Adjust semantics.
* tests/commandtest.c (test1): Test it.
* daemon/remote.c (remoteDispatchAuthPolkit): Adjust callers.
* src/access/viraccessdriverpolkit.c (virAccessDriverPolkitCheck):
Likewise.
* src/fdstream.c (virFDStreamCloseInt): Likewise.
* src/lxc/lxc_process.c (virLXCProcessStart): Likewise.
* src/qemu/qemu_command.c (qemuCreateInBridgePortWithHelper):
Likewise.
* src/xen/xen_driver.c (xenUnifiedXendProbe): Simplify.
* tests/reconnect.c (mymain): Likewise.
* tests/statstest.c (mymain): Likewise.
* src/bhyve/bhyve_process.c (virBhyveProcessStart)
(virBhyveProcessStop): Don't overwrite virCommand error.
* src/libvirt.c (virConnectAuthGainPolkit): Likewise.
* src/openvz/openvz_driver.c (openvzDomainGetBarrierLimit)
(openvzDomainSetBarrierLimit): Likewise.
* src/util/virebtables.c (virEbTablesOnceInit): Likewise.
* src/util/viriptables.c (virIpTablesOnceInit): Likewise.
* src/util/virnetdevveth.c (virNetDevVethCreate): Fix debug
message.
* src/qemu/qemu_capabilities.c (virQEMUCapsInitQMP): Add comment.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSINodeUpdate): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

These timeout values make librados/librbd return -ETIMEDOUT when a
operation is blocking due to a failing/unreachable Ceph cluster.

By having the operations time out libvirt will not block.

With this information it's easier for the user to debug what is
going wrong.

There has been a new field introduced in iscsiadm --mode session
output [1], but our regex only expects four fields. This breaks
startup of iscsi pools:
error: Failed to start pool iscsi
error: internal error: cannot find session

Fix this by ignoring anything after the fourth field.

https://bugzilla.redhat.com/show_bug.cgi?id=1067173

[1] https://github.com/mikechristie/open-iscsi/commit/181af9a

The internal pools were an idea in one of the first iterations of the
gluster series, which we decided not to use. Somehow the patch still
got pushed. Remove it as the internal flag isn't needed.

This reverts commit 362da820.