A future patch wants to create disk definitions with non-zero
default contents; to avoid crashes, all callers that allocate
a disk definition should go through a common point.

I found allocation points by looking for any code that increments
ndisks, as well as any matches for ALLOC.*disk.  Most places that
modified ndisks were covered by the parse from XML to domain/device
definition by initial domain creation or device hotplug; I also
hand-checked all drivers that generate a device struct on the
fly during getXMLDesc.

* src/conf/domain_conf.h (virDomainDiskDefNew): New prototype.
* src/conf/domain_conf.c (virDomainDiskDefNew): New function.
(virDomainDiskDefParseXML): Use it.
* src/parallels/parallels_driver.c (parallelsAddHddInfo):
Likewise.
* src/qemu/qemu_command.c (qemuParseCommandLine): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc): Likewise.
* src/vmx/vmx.c (virVMXParseDisk): Likewise.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenParseSxpr):
Likewise.
* src/xenxs/xen_xm.c (xenParseXM): Likewise.
* src/libvirt_private.syms (domain_conf.h): Export it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Openstack uses (or will start to using) CPU info from the
capabilities XML. So this section is expanded, added CPU info
about arch, type and info about number of cores, sockets and threads.
Signed-off-by: NEric Blake <eblake@redhat.com>

Openstack Nova (starting at Icehouse release) requires this function
to start VM.
Signed-off-by: NEric Blake <eblake@redhat.com>

OpenStack Nova requires this function
to start VM instance. Cpumask info is obtained via prlctl utility.
Unlike KVM, Parallels Cloud Server is unable to set cpu affinity
mask for every VCpu. Mask is unique for all VCpu. You can set it
using 'prlctl set <vm_id|vm_name> --cpumask <{n[,n,n1-n2]|all}>'
command. For example, 'prlctl set SomeDomain --cpumask 0,1,5-7'
would set this mask to yy---yyy.
Signed-off-by: NEric Blake <eblake@redhat.com>

Each VM consists of a set of files in PCS: config, hard
disk images, log file, memory dump. All these files are stored
in a per-vm directory. When we create a new VM, we can ether specify
path to the VM or create the VM in a default path
(<default path>/<vm name>.pvm). This default path can be configured
with command
prlsrvctl user set --def-vm-home <path> command.

Currenty parallels driver creates VM in the same place, where first
hard disk is located. Let's change this logic and create VMs in
the default path. It will be much clearer and allow us to create
VMs without hard disks.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Disks support in this driver was implemented with an assumption,
that disk images can't be created by hand, without VM. So
complex storage driver was implemented with workaround.

This is not true, we can create new disks using ploop tool.
So the first step to reimplement disks support in parallels
driver is to do not use information from the storage driver,
until we will implement VIR_STORAGE_TYPE_VOLUME disks.

So after this patch disks can be added in the same way as
in any other driver: you create a disk image and then add
an entry to the XML definition of the domain with path to that
image file, for example:

<disk type='file' device='disk'>
  <driver type='ploop'/>
  <source file='/storage/harddisk1.hdd'/>
  <target dev='sda' bus='sata'/>
  <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

This patch makes parallels storage driver useless, but I'll fix it
later. Now you can create an image by hand, using ploop tool,
and then add it to some domain.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Set file format in virDomainDef structure to produce correct
XML in virDomainGetXMLDesc function.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Add functions parallelsIsAlive, parallelsIsEncrypted,
parallelsIsSecure which are very simple to implement, but
may be required by some libvirt users. Almost all other
drivers have these functions.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

There is a problem with function parallelsDomainDefineXML. If we
are defining a new domain, then we need to do 2 things: aclually
create a VM in PCS and add new domain to the cached list of domains
_parallelsConn.domains.

This is done in the function parallelsLoadDomains. So call to
virDomainObjListAdd will return a error, because a domain
with the same name and id will already be in the list.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

I added this code year ago, instead of implementing ability
to change VNC configuration, which was not trivial, I added
extra call to prlctl, which sets up VNC with auto port, despite
VNC configuration given by a user.

Let's remove this hack, because, first, it doesn't work on the
latest Parallels Cloud Server release (you have to either specify
--vnc-nopasswd option or password). And also has problem with
error handling. If second call to prlctl fails, VM, created by
first call to prlctl, will not be removed.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

virDomainDef.features became an array, so now we can't simply
compare one features variable to another. We need to compare
each each element from the array.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

One of the features of qcow2 is that a wrapper file can have
more capacity than its backing file from the guest's perspective;
what's more, sparse files make tracking allocation of both
the active and backing file worthwhile.  As such, it makes
more sense to show allocation numbers for each file in a chain,
and not just the top-level file.  This sets up the fields for
the tracking, although it does not modify XML to display any
new information.

* src/util/virstoragefile.h (_virStorageSource): Add fields.
* src/conf/storage_conf.h (_virStorageVolDef): Drop redundant
fields.
* src/storage/storage_backend.c (virStorageBackendCreateBlockFrom)
(createRawFile, virStorageBackendCreateQemuImgCmd)
(virStorageBackendCreateQcowCreate): Update clients.
* src/storage/storage_driver.c (storageVolDelete)
(storageVolCreateXML, storageVolCreateXMLFrom, storageVolResize)
(storageVolWipeInternal, storageVolGetInfo): Likewise.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget)
(virStorageBackendFileSystemRefresh)
(virStorageBackendFileSystemVolResize)
(virStorageBackendFileSystemVolRefresh): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalMakeVol)
(virStorageBackendLogicalCreateVol): Likewise.
* src/storage/storage_backend_scsi.c
(virStorageBackendSCSINewLun): Likewise.
* src/storage/storage_backend_mpath.c
(virStorageBackendMpathNewVol): Likewise.
* src/storage/storage_backend_rbd.c
(volStorageBackendRBDRefreshVolInfo)
(virStorageBackendRBDCreateImage): Likewise.
* src/storage/storage_backend_disk.c
(virStorageBackendDiskMakeDataVol)
(virStorageBackendDiskCreateVol): Likewise.
* src/storage/storage_backend_sheepdog.c
(virStorageBackendSheepdogBuildVol)
(virStorageBackendSheepdogParseVdiList): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Likewise.
* src/conf/storage_conf.c (virStorageVolDefFormat)
(virStorageVolDefParseXML): Likewise.
* src/test/test_driver.c (testOpenVolumesForPool)
(testStorageVolCreateXML, testStorageVolCreateXMLFrom)
(testStorageVolDelete, testStorageVolGetInfo): Likewise.
* src/esx/esx_storage_backend_iscsi.c (esxStorageVolGetXMLDesc):
Likewise.
* src/esx/esx_storage_backend_vmfs.c (esxStorageVolGetXMLDesc)
(esxStorageVolCreateXML): Likewise.
* src/parallels/parallels_driver.c (parallelsAddHddByVolume):
Likewise.
* src/parallels/parallels_storage.c (parallelsDiskDescParseNode)
(parallelsStorageVolDefineXML, parallelsStorageVolCreateXMLFrom)
(parallelsStorageVolDefRemove, parallelsStorageVolGetInfo):
Likewise.
* src/vbox/vbox_tmpl.c (vboxStorageVolCreateXML)
(vboxStorageVolGetXMLDesc): Likewise.
* tests/storagebackendsheepdogtest.c (test_vdi_list_parser):
Likewise.
* src/phyp/phyp_driver.c (phypStorageVolCreateXML): Likewise.

A continuation of the migration of disk details to virstoragefile.
This patch moves a single enum, but converting the name has quite
a bit of fallout.

* src/conf/domain_conf.h (virDomainDiskType): Move...
* src/util/virstoragefile.h (virStorageType): ...and rename.
* src/bhyve/bhyve_command.c (bhyveBuildDiskArgStr)
(virBhyveProcessBuildLoadCmd): Update clients.
* src/conf/domain_conf.c (virDomainDiskSourceDefParse)
(virDomainDiskDefParseXML, virDomainDiskSourceDefFormatInternal)
(virDomainDiskDefFormat, virDomainDiskGetActualType)
(virDomainDiskDefForeachPath, virDomainDiskSourceIsBlockType):
Likewise.
* src/conf/snapshot_conf.h (_virDomainSnapshotDiskDef): Likewise.
* src/conf/snapshot_conf.c (virDomainSnapshotDiskDefParseXML)
(virDomainSnapshotAlignDisks, virDomainSnapshotDiskDefFormat):
Likewise.
* src/esx/esx_driver.c (esxAutodetectSCSIControllerModel)
(esxDomainDefineXML): Likewise.
* src/locking/domain_lock.c (virDomainLockManagerAddDisk):
Likewise.
* src/lxc/lxc_controller.c
(virLXCControllerSetupLoopDeviceDisk)
(virLXCControllerSetupNBDDeviceDisk)
(virLXCControllerSetupLoopDevices, virLXCControllerSetupDisk):
Likewise.
* src/parallels/parallels_driver.c (parallelsGetHddInfo):
Likewise.
* src/phyp/phyp_driver.c (phypDiskType): Likewise.
* src/qemu/qemu_command.c (qemuGetDriveSourceString)
(qemuDomainDiskGetSourceString, qemuBuildDriveStr)
(qemuBuildCommandLine, qemuParseCommandLineDisk)
(qemuParseCommandLine): Likewise.
* src/qemu/qemu_conf.c (qemuCheckSharedDevice)
(qemuTranslateDiskSourcePool)
(qemuTranslateSnapshotDiskSourcePool): Likewise.
* src/qemu/qemu_domain.c (qemuDomainDeviceDefPostParse)
(qemuDomainDetermineDiskChain): Likewise.
* src/qemu/qemu_driver.c (qemuDomainGetBlockInfo)
(qemuDomainSnapshotPrepareDiskExternalBackingInactive)
(qemuDomainSnapshotPrepareDiskExternalBackingActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayActive)
(qemuDomainSnapshotPrepareDiskExternalOverlayInactive)
(qemuDomainSnapshotPrepareDiskInternal)
(qemuDomainSnapshotPrepare)
(qemuDomainSnapshotCreateSingleDiskActive): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsSafe): Likewise.
* src/security/security_apparmor.c
(AppArmorRestoreSecurityImageLabel)
(AppArmorSetSecurityImageLabel): Likewise.
* src/security/security_dac.c (virSecurityDACSetSecurityImageLabel)
(virSecurityDACRestoreSecurityImageLabelInt)
(virSecurityDACSetSecurityAllLabel): Likewise.
* src/security/security_selinux.c
(virSecuritySELinuxRestoreSecurityImageLabelInt)
(virSecuritySELinuxSetSecurityImageLabel)
(virSecuritySELinuxSetSecurityAllLabel): Likewise.
* src/storage/storage_backend.c (virStorageFileBackendForType):
Likewise.
* src/storage/storage_backend_fs.c (virStorageFileBackendFile)
(virStorageFileBackendBlock): Likewise.
* src/storage/storage_backend_gluster.c
(virStorageFileBackendGluster): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc, vboxAttachDrives)
(vboxDomainAttachDeviceImpl, vboxDomainDetachDevice): Likewise.
* src/vmware/vmware_conf.c (vmwareVmxPath): Likewise.
* src/vmx/vmx.c (virVMXParseDisk, virVMXFormatDisk)
(virVMXFormatFloppy): Likewise.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenParseSxpr)
(xenFormatSxprDisk): Likewise.
* src/xenxs/xen_xm.c (xenParseXM, xenFormatXMDisk): Likewise.
* tests/securityselinuxlabeltest.c (testSELinuxLoadDef):
Likewise.
* src/libvirt_private.syms (domain_conf.h): Move symbols...
(virstoragefile.h): ...as appropriate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Currently, <cputune><shares>0</shares></cputune> is treated
as if it were not specified.

Treat is as a valid value if it was explicitly specified
and write it to the cgroups.

Part of a series of cleanups to use new accessor methods.

* src/parallels/parallels_driver.c (parallelsGetHddInfo)
(parallelsAddHdd, parallelsApplyDisksParams, parallelsCreateVm):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Ensure that all APIs which list domain objects filter
them against the access control system.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

These all existed before virfile.c was created, and for some reason
weren't moved.

This is mostly straightfoward, although the syntax rule prohibiting
write() had to be changed to have an exception for virfile.c instead
of virutil.c.

This movement pointed out that there is a function called
virBuildPath(), and another almost identical function called
virFileBuildPath(). They really should be a single function, which
I'll take care of as soon as I figure out what the arglist should look
like.

The individual hypervisor drivers were directly referencing
APIs in src/nodeinfo.c in their virDriverPtr struct. Separate
these methods, so there is always a wrapper in the hypervisor
driver. This allows the unused virConnectPtr args to be
removed from the nodeinfo.c file. Again this will ensure that
ACL checks will only be performed on invocations that are
directly associated with public API usage.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the virGetHostname() API has a bogus virConnectPtr
parameter. This is because virtualization drivers directly
reference this API in their virDriverPtr tables, tieing its
API design to the public virConnectGetHostname API design.

This also causes problems for access control checks since
these must only be done for invocations from the public
API, not internal invocation.

Remove the bogus virConnectPtr parameter, and make each
hypervisor driver provide a dedicated function for the
driver API impl. This will allow access control checks
to be easily inserted later.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

Ensure that all drivers implementing public APIs use a
naming convention for their implementation that matches
the public API name.

eg for the public API   virDomainCreate make sure QEMU
uses qemuDomainCreate and not qemuDomainStart
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Ensure that the driver struct field names match the public
API names. For an API virXXXX we must have a driver struct
field xXXXX. ie strip the leading 'vir' and lowercase any
leading uppercase letters.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Detected by a simple Shell script:

for i in $(git ls-files -- '*.[ch]'); do
    awk 'BEGIN {
        fail=0
    }
    /# *include.*\.h/{
        match($0, /["<][^">]*[">]/)
        arr[substr($0, RSTART+1, RLENGTH-2)]++
    }
    END {
        for (key in arr) {
            if (arr[key] > 1) {
                fail=1
                printf("%d %s\n", arr[key], key)
            }
        }
        if (fail == 1)
            exit 1
    }' $i

    if test $? != 0; then
        echo "Duplicate header(s) in $i"
    fi
done;

A later patch will add the syntax-check to avoid duplicate
headers.

This patch refactors various places to allow removing of the
defaultConsoleTargetType callback from the virCaps structure.

A new console character device target type is introduced -
VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_NONE - to mark that no type was
specified in the XML. This type is at the end converted to the standard
VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL. Other types that are
different from this default have to be processed separately in the
device post parse callback.

Use the virDomainXMLConf structure to hold this data and tweak the code
to avoid semantic change.

Without configuration the KVM mac prefix is used by default. I chose it
as it's in the privately administered segment so it should be usable for
any purposes.

Move the xmlopt and caps arguments to the end of the argument list.

This patch adds instrumentation that will allow hypervisor drivers to
fill and validate domain and device definitions after parsed by the XML
parser.

With this patch, after the XML is parsed, a callback to the driver is
issued requesting to fill and validate driver specific details of the
configuration. This allows to use sensible defaults and checks on a per
driver basis at the time the XML is parsed.

Two callback pointers are stored in the new virDomainXMLConf object:
* virDomainDeviceDefPostParseCallback (devicesPostParseCallback)
  - called for a single device parsed and for every single device in a
    domain config. A virDomainDeviceDefPtr is passed along with the
    domain definition and virCaps.

* virDomainDefPostParseCallback, (domainPostParseCallback)
  - A callback that is meant to process the domain config after it's
  parsed.  A virDomainDefPtr is passed along with virCaps.

Both types of callbacks support arbitrary opaque data passed for the
callback functions.

Errors may be reported in those callbacks resulting in a XML parsing
failure.

This patch is the result of running:

for i in $(git ls-files | grep -v html | grep -v \.po$ ); do
  sed -i -e "s/virDomainXMLConf/virDomainXMLOption/g" -e "s/xmlconf/xmlopt/g" $i
done

and a few manual tweaks.

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

To enable virCapabilities instances to be reference counted,
turn it into a virObject. All cases of virCapabilitiesFree
turn into virObjectUnref
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The duplicate VM checking should be done atomically with
virDomainObjListAdd, so shoud not be a separate function.
Instead just use flags to indicate what kind of checks are
required.

This pair, used in virDomainCreateXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainRestoreFlags:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, true)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_LIVE |
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainDefineXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 0) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   0, NULL)))
     goto cleanup;

As a step towards making virDomainObjList thread-safe turn it
into an opaque virObject, preventing any direct access to its
internals.

As part of this a new method virDomainObjListForEach is
introduced to replace all existing usage of virHashForEach