A continuation of the migration of disk details to virstoragefile.
This patch moves a single enum, but converting the name has quite
a bit of fallout.

* src/conf/domain_conf.h (virDomainDiskType): Move...
* src/util/virstoragefile.h (virStorageType): ...and rename.
* src/bhyve/bhyve_command.c (bhyveBuildDiskArgStr)
(virBhyveProcessBuildLoadCmd): Update clients.
* src/conf/domain_conf.c (virDomainDiskSourceDefParse)
(virDomainDiskDefParseXML, virDomainDiskSourceDefFormatInternal)
(virDomainDiskDefFormat, virDomainDiskGetActualType)
(virDomainDiskDefForeachPath, virDomainDiskSourceIsBlockType):
Likewise.
* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefParseXML)
(virDomainSnapshotAlignDisks, virDomainSnapshotDiskDefFormat):
Likewise.
* src/esx/esx_driver.c (esxAutodetectSCSIControllerModel)
(esxDomainDefineXML): Likewise.
* src/locking/domain_lock.c (virDomainLockManagerAddDisk):
Likewise.
* src/lxc/lxc_controller.c
(virLXCControllerSetupLoopDeviceDisk)
(virLXCControllerSetupNBDDeviceDisk)
(virLXCControllerSetupLoopDevices, virLXCControllerSetupDisk):
Likewise.
* src/parallels/parallels_driver.c (parallelsGetHddInfo):
Likewise.
* src/phyp/phyp_driver.c (phypDiskType): Likewise.
* src/qemu/qemu_command.c (qemuGetDriveSourceString)
(qemuDomainDiskGetSourceString, qemuBuildDriveStr)
(qemuBuildCommandLine, qemuParseCommandLineDisk)
(qemuParseCommandLine): Likewise.
* src/qemu/qemu_conf.c (qemuCheckSharedDevice)
(qemuTranslateDiskSourcePool)
(qemuTranslateSnapshotDiskSourcePool): Likewise.
* src/qemu/qemu_domain.c (qemuDomainDeviceDefPostParse)
(qemuDomainDetermineDiskChain): Likewise.
* src/qemu/qemu_driver.c (qemuDomainGetBlockInfo)
(qemuDomainSnapshotPrepareDiskExternalBackingInactive)
(qemuDomainSnapshotPrepareDiskExternalBackingActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayInactive)
(qemuDomainSnapshotPrepareDiskInternal)
(qemuDomainSnapshotPrepare)
(qemuDomainSnapshotCreateSingleDiskActive): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsSafe): Likewise.
* src/security/security_apparmor.c
(AppArmorRestoreSecurityImageLabel)
(AppArmorSetSecurityImageLabel): Likewise.
* src/security/security_dac.c (virSecurityDACSetSecurityImageLabel)
(virSecurityDACRestoreSecurityImageLabelInt)
(virSecurityDACSetSecurityAllLabel): Likewise.
* src/security/security_selinux.c
(virSecuritySELinuxRestoreSecurityImageLabelInt)
(virSecuritySELinuxSetSecurityImageLabel)
(virSecuritySELinuxSetSecurityAllLabel): Likewise.
* src/storage/storage_backend.c (virStorageFileBackendForType):
Likewise.
* src/storage/storage_backend_fs.c (virStorageFileBackendFile)
(virStorageFileBackendBlock): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageFileBackendGluster): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc, vboxAttachDrives)
(vboxDomainAttachDeviceImpl, vboxDomainDetachDevice): Likewise.
* src/vmware/vmware_conf.c (vmwareVmxPath): Likewise.
* src/vmx/vmx.c (virVMXParseDisk, virVMXFormatDisk)
(virVMXFormatFloppy): Likewise.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenParseSxpr)
(xenFormatSxprDisk): Likewise.
* src/xenxs/xen_xm.c (xenParseXM, xenFormatXMDisk): Likewise.
* tests/securityselinuxlabeltest.c (testSELinuxLoadDef):
Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Continuing the refactoring of host-side storage descriptions out
of conf/domain_conf and into util/virstoragefile, this patch
focuses on details about a host name/port/transport as used by
a network storage volume.

* src/conf/domain_conf.h (virDomainDiskProtocolTransport)
(virDomainDiskHostDef, virDomainDiskHostDefClear)
(virDomainDiskHostDefFree, virDomainDiskHostDefCopy): Move...
* src/util/virstoragefile.h (virStorageNetHostTransport)
(virStorageNetHostDef, virStorageNetHostDefClear)
(virStorageNetHostDefFree, virStorageNetHostDefCopy): ...here,
with better names.
* src/util/virstoragefile.c (virStorageNetHostDefClear)
(virStorageNetHostDefFree, virStorageNetHostDefCopy): Moved from...
* src/conf/domain_conf.c (virDomainDiskHostDefClear)
(virDomainDiskHostDefFree, virDomainDiskHostDefCopy): ...here.
(virDomainDiskSourceDefClear, virDomainDiskSourceDefParse)
(virDomainDiskSourceDefFormatInternal): Adjust callers.
* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefClear):
Likewise.
* src/qemu/qemu_command.c (qemuAddRBDHost)
(qemuParseDriveURIString, qemuParseNBDString)
(qemuBuildNetworkDriveURI, qemuParseCommandLineDisk)
(qemuParseCommandLine, qemuGetDriveSourceString): Likewise.
* src/qemu/qemu_command.h: Likewise.
* src/qemu/qemu_conf.c (qemuAddISCSIPoolSourceHost)
(qemuTranslateDiskSourcePool): Likewise.
* src/qemu/qemu_driver.c
(qemuDomainSnapshotCreateSingleDiskActive)
(qemuDomainSnapshotUndoSingleDiskActive): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageFileBackendGlusterInit): Likewise.
* src/storage/storage_driver.c (virStorageFileFree)
(virStorageFileInitInternal): Likewise.
* src/storage/storage_driver.h (_virStorageFile): Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Part of a series of cleanups to use new accessor methods.

Several places in domain_conf.c still open-code raw field access,
but that code will be touched later with the diskDef struct split
so I'm avoiding churn here.

* src/conf/domain_audit.c (virDomainAuditStart): Use accessors.
* src/conf/domain_conf.c (virDomainDiskIndexByName)
(virDomainDiskPathByName, virDomainDiskDefForeachPath)
(virDomainDiskSourceIsBlockType): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotAlignDisks):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

While running domainsnapshotxml2xmltest, it was found that valgrind pointed out
the following memory leak:

==32176== 42 (32 direct, 10 indirect) bytes in 1 blocks are definitely lost in loss record 42 of 66
==32176==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==32176==    by 0x4A06B62: realloc (vg_replace_malloc.c:662)
==32176==    by 0x4C65A07: virReallocN (viralloc.c:243)
==32176==    by 0x4C65B2E: virExpandN (viralloc.c:292)
==32176==    by 0x4C65E30: virInsertElementsN (viralloc.c:434)
==32176==    by 0x4CD71F3: virDomainDiskSourceDefParse (domain_conf.c:5078)
==32176==    by 0x4CF6EF4: virDomainSnapshotDefParseNode (snapshot_conf.c:151)
==32176==    by 0x4CF7314: virDomainSnapshotDefParseString (snapshot_conf.c:410)
==32176==    by 0x41FB8D: testCompareXMLToXMLHelper (domainsnapshotxml2xmltest.c:100)
==32176==    by 0x420FD1: virtTestRun (testutils.c:199)
==32176==    by 0x41F859: mymain (domainsnapshotxml2xmltest.c:222)
==32176==    by 0x42174D: virtTestMain (testutils.c:782)
==32176==
... and one more.

All leading spaces in domain snapshot xml format functions have been
replaced with appropriate calls to virBufferAdjustIndent(). This will
make it easier to call other similarly fixed format functions
(e.g. domain device format functions).

Many of the domain xml format functions (including all of the device
format functions) had hard-coded spaces, which made for incorrect
indentation when those functions were called in a different context
(for example, commit 2122cf39 added <interface> XML into the document
provided to a network hook script, and in this case it should have
been indented by 2 spaces, but was instead indented by 6 spaces).

To make it possible to insert a properly indented device anywhere into
an XML document, this patch removes hardcoded spaces from the
formatting functions, and calls virBufferAdjustIndent() at appropriate
places instead. (a regex search of domain_conf.c was done to assure
that all occurrences of hardcoded spaces were removed).

virDomainDiskSourceDefFormatInternal() is also called from
snapshot_conf.c, so two virBufferAdjustIndent() calls were temporarily
added around that call - those functions will have hardcoded spaces
removed in a separate patch.

This could cause some conflicts when backporting future changes to the
formatting functions to older branches, but fortunately the changes
are almost all trivial, so conflict resolution will be obvious.

All the data for getting the actual type is present in the snapshot
config. There is no need to have this function private to the qemu
driver and it will be re-used later in other parts of libvirt

Add support for specifying various types when doing snapshots. This will
later allow to do snapshots on network backed volumes. Disks of type
'volume' are not supported by snapshots (yet).

Also amend the test suite to check parsing of the various new disk
types that can now be specified.

Currently, during XML parsing, when a call to a FromString() function to
get an enum value fails, the error which is reported is either
VIR_ERR_CONFIG_UNSUPPORTED, VIR_ERR_INTERNAL_ERROR or VIR_ERR_XML_ERROR.

This commit makes such conversion failures consistently return
VIR_ERR_CONFIG_UNSUPPORTED.

When changing the parsing and formatting functions in commit
43f2ccdc I forgot to update the qemu
disk alignment function for snapshots that automatically adds snapshot
configs for disks that were not mentioned in the XML. The function
allocated a new disk snapshot definition but did not correctly
initialize the snapshot disk source type variable. This resulted into
the disks considered as block devices and invalid XML was generated.

Reported by John Ferlan.

Consider the following valid snapshot XML as the <driver> element is
allowed to be empty in the domainsnapshot.rng schema:

$ cat snap.xml
<domainsnapshot>
  <disks>
    <disk name='vda' snapshot='external'>
      <source file='/tmp/foo'/>
      <driver/>
    </disk>
  </disks>
</domainsnapshot>

produces the following error:

$ virsh snapshot-create domain snap.xml
error: internal error: unknown disk snapshot driver '(null)'

The driver type is parsed as NULL from the XML as the attribute is not
present and then directly used to produce the error message.

With this patch the attempt to parse the driver type is skipped if not
present to avoid changing the schema to forbid the empty driver element.

Disk source elements for snapshots were using separate code from our
config parser. As snapshots can be stored on more than just regular
files, we will need the universal parser to allow us to expose a variety
of snapshot disk targets. This patch reuses the config parsers and
formatters to do the job.

This initial support only changes the code without any visible XML
change.

'const fooPtr' is the same as 'foo * const' (the pointer won't
change, but it's contents can).  But in general, if an interface
is trying to be const-correct, it should be using 'const foo *'
(the pointer is to data that can't be changed).

Fix up remaining offenders in src/conf, and their fallout.

* src/conf/snapshot_conf.h (virDomainSnapshotAssignDef)
(virDomainSnapshotFindByName): Drop attempt at const.
* src/conf/interface_conf.h (virInterfaceObjIsActive)
(virInterfaceDefFormat): Use intended type.
(virInterfaceFindByMACString, virInterfaceFindByName)
(virInterfaceAssignDef, virInterfaceRemove): Drop attempt at
const.
* src/conf/network_conf.h (virNetworkObjIsActive)
(virNetworkDefFormat, virNetworkDefForwardIf)
(virNetworkDefGetIpByIndex, virNetworkIpDefPrefix)
(virNetworkIpDefNetmask): Use intended type.
(virNetworkFindByUUID, virNetworkFindByName, virNetworkAssignDef)
(virNetworkObjAssignDef, virNetworkRemoveInactive)
(virNetworkBridgeInUse, virNetworkSetBridgeName)
(virNetworkAllocateBridge): Drop attempt at const.
* src/conf/netdev_vlan_conf.h (virNetDevVlanFormat): Make
const-correct.
* src/conf/node_device_conf.h (virNodeDeviceHasCap)
(virNodeDeviceDefFormat): Use intended type.
(virNodeDeviceFindByName, virNodeDeviceFindBySysfsPath)
(virNodeDeviceAssignDef, virNodeDeviceObjRemove)
(virNodeDeviceGetParentHost): Drop attempt at const.
* src/conf/secret_conf.h (virSecretDefFormat): Use intended type.
* src/conf/snapshot_conf.c (virDomainSnapshotAssignDef)
(virDomainSnapshotFindByName): Fix fallout.
* src/conf/interface_conf.c (virInterfaceBridgeDefFormat)
(virInterfaceBondDefFormat, virInterfaceVlanDefFormat)
(virInterfaceProtocolDefFormat, virInterfaceDefDevFormat)
(virInterfaceDefFormat, virInterfaceFindByMACString)
(virInterfaceFindByName, virInterfaceAssignDef)
(virInterfaceRemove): Likewise.
* src/conf/network_conf.c
(VIR_ENUM_IMPL, virNetworkFindByName, virNetworkObjAssignDef)
(virNetworkAssignDef, virNetworkRemoveInactive)
(virNetworkDefGetIpByIndex, virNetworkIpDefPrefix)
(virNetworkIpDefNetmask, virNetworkDHCPHostDefParseXML)
(virNetworkIpDefFormat, virNetworkRouteDefFormat)
(virPortGroupDefFormat, virNetworkForwardNatDefFormat)
(virNetworkDefFormatInternal, virNetworkBridgeInUse)
(virNetworkAllocateBridge, virNetworkSetBridgeName)
(virNetworkDNSDefFormat, virNetworkDefFormat): Likewise.
* src/conf/netdev_vlan_conf.c (virNetDevVlanFormat): Likewise.
* src/conf/node_device_conf.c (virNodeDeviceHasCap)
(virNodeDeviceFindBySysfsPath, virNodeDeviceFindByName)
(virNodeDeviceAssignDef, virNodeDeviceObjRemove)
(virNodeDeviceDefFormat, virNodeDeviceGetParentHost): Likewise.
* src/conf/secret_conf.c (virSecretDefFormatUsage)
(virSecretDefFormat): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

The virDomainSnapshotDefParse method assigned to def->ndisks
before allocating def->disks. Thus if an OOM occurred, the
cleanup code would access out of bounds.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Similar to how other objects arrange their parse APIs. This will be
used by the test driver.

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Just to reduce the indentation levels. Remove the unneeded
NULL check for disk->file, as virBufferEscapeString doesn't
print anything with NULL arguments.

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

Move the xmlopt and caps arguments to the end of the argument list.

This patch is the result of running:

for i in $(git ls-files | grep -v html | grep -v \.po$ ); do
  sed -i -e "s/virDomainXMLConf/virDomainXMLOption/g" -e "s/xmlconf/xmlopt/g" $i
done

and a few manual tweaks.

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

When a disk-only snapshot is requested the domain is treated as if it
was offline. This forbids to mix memory checkpoints with the DISK_ONLY
flag.

This patch improves the error message and mentions the restriction in
the virsh man page.

Allow to use definition objects with this predicate function.

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If allocation of names fails, list is uninitialized.

If it's negative, this might result in a request to allocate lots of
memory.

Relatively straight-forward.  And since qemu was already using
VIR_DOMAIN_SNAPSHOT_FILTERS_ALL, with 6 different APIs all calling
into this common code, I've instantly added all 5 flags to 6 APIs.

* src/conf/snapshot_conf.h (VIR_DOMAIN_SNAPSHOT_FILTERS_ALL):
Enable new filters.
* src/conf/snapshot_conf.c (virDomainSnapshotObjListGetNames):
Prep the new flags.
(virDomainSnapshotObjListCopyNames): Actually do the filtering.