When testing SASL authentication over TLS with
virsh -c qemu+tls:///system list --all
I got this valgrind trace after entering wrong credentials:

==30540== 26,903 (88 direct, 26,815 indirect) bytes in 1 blocks are definitely lost in loss record 289 of 293
==30540==    at 0x4A081D4: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30540==    by 0x4C7379A: virAllocVar (viralloc.c:558)
==30540==    by 0x4CBC178: virObjectNew (virobject.c:190)
==30540==    by 0x4CBC329: virObjectLockableNew (virobject.c:216)
==30540==    by 0x4E2D003: virNetTLSContextNew (virnettlscontext.c:719)
==30540==    by 0x4E2DC3F: virNetTLSContextNewPath (virnettlscontext.c:930)
==30540==    by 0x4E2DD5B: virNetTLSContextNewClientPath (virnettlscontext.c:957)
==30540==    by 0x4DDB618: doRemoteOpen (remote_driver.c:627)
==30540==    by 0x4DDC8BA: remoteConnectOpen (remote_driver.c:1031)
==30540==    by 0x4D8595F: do_open (libvirt.c:1239)
==30540==    by 0x4D863F3: virConnectOpenAuth (libvirt.c:1481)
==30540==    by 0x12762B: vshReconnect (virsh.c:337)
==30540==    by 0x12C9B0: vshInit (virsh.c:2470)
==30540==    by 0x12E9A5: main (virsh.c:3338)

You'd think I'd learn to actually COMMIT my working tree
between testing that a last-minute fix compiles and pushing.

* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Typo fix.
Signed-off-by: NEric Blake <eblake@redhat.com>

Putting together pieces from previous patches, it is now possible
for 'virsh vol-dumpxml --pool gluster volname' to report metadata
about a qcow2 file stored on gluster.  The backing file is still
treated as raw; to fix that, more patches are needed to make the
storage backing chain analysis recursive rather than halting at
a network protocol name, but that work will not need any further
calls into libgfapi so much as just reusing this code, and that
should be the only code outside of the storage driver that needs
any help from libgfapi.  Any additional use of libgfapi within
libvirt should only be needed for implementing storage pool APIs
such as volume creation or resizing, where backing chain analysis
should be unaffected.

* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterReadHeader): New helper function.
(virStorageBackendGlusterRefreshVol): Probe non-raw files.
Signed-off-by: NEric Blake <eblake@redhat.com>

With this patch, dangling and looping symlinks are silently
ignored, while links to files and directories are treated the
same as the underlying file or directory.  This is the same
behavior as both 'directory' and 'netfs' pools.

* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Treat symlinks similar to
directory and netfs pools.
Signed-off-by: NEric Blake <eblake@redhat.com>

We already had code for handling allocation different than
capacity for sparse files; we just had to wire it up to be
used when inspecting gluster images.

* src/storage/storage_backend.c
(virStorageBackendUpdateVolTargetInfoFD): Handle no fd.
* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Handle sparse files.
Signed-off-by: NEric Blake <eblake@redhat.com>

Take advantage of the previous patch's addition of 'netdir' as
a distinct volume type, to expose rather than silently skip
directories embedded in a gluster pool.  Also serves as an XML
validation for the previous patch.

* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Don't skip directories.
* tests/storagevolxml2xmltest.c (mymain): Add test.
* tests/storagevolxml2xmlin/vol-gluster-dir.xml: New file.
* tests/storagevolxml2xmlout/vol-gluster-dir.xml: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

In the 'directory' and 'netfs' storage pools, a user can see
both 'file' and 'dir' storage volume types, to know when they
can descend into a subdirectory.  But in a network-based storage
pool, such as the upcoming 'gluster' pool, we use 'network'
instead of 'file', and did not have any counterpart for a
directory until this patch.  Adding a new volume type
'network-dir' is better than reusing 'dir', because it makes
it clear that the only way to access 'network' volumes within
that container is through the network mounting (leaving 'dir'
for something accessible in the local file system).

* include/libvirt/libvirt.h.in (virStorageVolType): Expand enum.
* docs/formatstorage.html.in: Document it.
* docs/schemasa/storagevol.rng (vol): Allow new value.
* src/conf/storage_conf.c (virStorageVol): Use new value.
* src/qemu/qemu_command.c (qemuBuildVolumeString): Fix client.
* src/qemu/qemu_conf.c (qemuTranslateDiskSourcePool): Likewise.
* tools/virsh-volume.c (vshVolumeTypeToString): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemVolDelete): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Actually put gfapi to use, by allowing the creation of a gluster
pool.  Right now, all volumes are treated as raw and directories
are skipped; further patches will allow peering into files to
allow for qcow2 files and backing chains, and reporting proper
volume allocation.  This implementation was tested against Fedora
19's glusterfs 3.4.1; it might be made simpler by requiring a
higher minimum, and/or require more hacks to work with a lower
minimum.

* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshPool): Initial implementation.
(virStorageBackendGlusterOpen, virStorageBackendGlusterClose)
(virStorageBackendGlusterRefreshVol): New helper functions.
Signed-off-by: NEric Blake <eblake@redhat.com>

We support gluster volumes in domain XML, so we also ought to
support them as a storage pool.  Besides, a future patch will
want to take advantage of libgfapi to handle the case of a
gluster device holding qcow2 rather than raw storage, and for
that to work, we need a storage backend that can read gluster
storage volume contents.  This sets up the framework.

Note that the new pool is named 'gluster' to match a
<disk type='network'><source protocol='gluster'> image source
already supported in a <domain>; it does NOT match the
<pool type='netfs'><source><target type='glusterfs'>,
since that uses a FUSE mount to a local file name rather than
a network name.

This and subsequent patches have been tested against glusterfs
3.4.1 (available on Fedora 19); there are likely bugs in older
versions that may prevent decent use of gfapi, so this patch
enforces the minimum version tested.  A future patch may lower
the minimum.  On the other hand, I hit at least two bugs in
3.4.1 that will be fixed in 3.5/3.4.2, where it might be worth
raising the minimum: glfs_readdir is nicer to use than
glfs_readdir_r [1], and glfs_fini should only return failure on
an actual failure [2].

[1] http://lists.gnu.org/archive/html/gluster-devel/2013-10/msg00085.html
[2] http://lists.gnu.org/archive/html/gluster-devel/2013-10/msg00086.html

* configure.ac (WITH_STORAGE_GLUSTER): New conditional.
* m4/virt-gluster.m4: new file.
* libvirt.spec.in (BuildRequires): Support gluster in spec file.
* src/conf/storage_conf.h (VIR_STORAGE_POOL_GLUSTER): New pool
type.
* src/conf/storage_conf.c (poolTypeInfo): Treat similar to
sheepdog and rbd.
(virStoragePoolDefFormat): Don't output target for gluster.
* src/storage/storage_backend_gluster.h: New file.
* src/storage/storage_backend_gluster.c: Likewise.
* po/POTFILES.in: Add new file.
* src/storage/storage_backend.c (backends): Register new type.
* src/Makefile.am (STORAGE_DRIVER_GLUSTER_SOURCES): Build new files.
* src/storage/storage_backend.h (_virStorageBackend): Documet
assumption.
Signed-off-by: NEric Blake <eblake@redhat.com>

I got annoyed at having to use both 'virsh vol-list $pool --details'
AND 'virsh vol-dumpxml $vol $pool' to learn if I had populated
the volume correctly.  Since two-thirds of the data present in
virStorageVolGetInfo() already appears in virStorageVolGetXMLDesc(),
this just adds the remaining piece of information, as:

<volume type='...'>
  ...
</volume>

* docs/formatstorage.html.in: Document new <volume type=...>.
* docs/schemas/storagevol.rng (vol): Add it to RelaxNG.
* src/conf/storage_conf.h (virStorageVolTypeToString): Declare.
* src/conf/storage_conf.c (virStorageVolTargetDefFormat): Output
the metatype.
(virStorageVolDefParseXML): Parse it, for unit tests.
* tests/storagevolxml2xmlout/vol-*.xml: Update tests to match.
Signed-off-by: NEric Blake <eblake@redhat.com>

The bus type IDE being enum Zero, the bus type on pseries system appears as IDE for all the -hda/-cdrom and for disk drives with if="none" type. Pseries platform needs this to appear as SCSI instead of IDE. The ide being not supported, the explicit requests for ide devices will return an error.
Signed-off-by: NShivaprasad G Bhat <sbhat@linux.vnet.ibm.com>

I didn't find any other instances with:
git grep '1\.1\.5'

* src/test/test_driver.c (testDriver): Tweak version info.
Signed-off-by: NEric Blake <eblake@redhat.com>

Makefile.am, vbox_V4_3.c and vbox_driver.c do regular
modifitions to support a new version of APIs.

vbox_tmpl.c basically fixes incompatibilities since 4.2.

The affected incompatibilities of 4.3 are:
* IMachine::Delete() has been renamed to IMachine::deleteConfig()
* IMedium::CreateBaseStorage() now accepts multiple variant values
* IDisplay::GetScreenResolution() now returns the display position
  in the guest
* IMachine now has multiple IUSBControllers and IUSBDeviceFilters
  handles USB device filters instead of (obsolete) IUSBController

This patch is tested on Mac OS X 10.8.5 and Fedora 19.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>

vbox_CAPI_v4_3.h is almost same as
sdk/bindings/xpcom/include/VBoxCAPI_v4_3.h of
http://download.virtualbox.org/virtualbox/4.3.2/VirtualBoxSDK-4.3.2-90405.zip,
but modified to fix preprocessor indentations by using cppi.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>

The USB-related code in vboxDomainGetXMLDesc is deeply nested and
difficult to add new code. So flatten it. To do so, the code is
pulled out from vboxDomainGetXMLDesc to make the function short
and to leaverage early return and goto for error handling.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>

This cleanup flattens deeply nested code.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>

Don't do duplicate work when getting pagesize.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

This nested job is canceled by the first ExitMonitor call (even though
it was not created by the corresponding EnterMonitor call), and
again in qemuMigrationPrepareAny if qemuProcessStart failed.
This can lead to a crash if the vm object was disposed of before calling
qemuDomainRemoveInactive:
0  ..62bc in virClassIsDerivedFrom (klass=0xdeadbeef,
   parent=0x7ffce4cdd270) at util/virobject.c:166
1 ..6666 in virObjectIsClass at util/virobject.c:362
2 ..66b4 in virObjectLock at util/virobject.c:314
3 ..477e in virDomainObjListRemove at conf/domain_conf.c:2359
4 ..7a64 in qemuDomainRemoveInactive at qemu/qemu_domain.c:2087
5 ..956c in qemuMigrationPrepareAny at qemu/qemu_migration.c:2469

This was added by commit e4e28220, exposed by 5a4c2374 and c7ac2519.

https://bugzilla.redhat.com/show_bug.cgi?id=1018267

https://bugzilla.redhat.com/show_bug.cgi?id=744967

If a domain is rebooting and a migrate API is called meanwhile we would
have to transfer the fakeReboot attribute to the destination in order to
prevent domain doing plain shutdown over there. We shouldn't try to do
anything clever about it other than documenting this as a known
limitation.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

$ touch /var/lib/libvirt/images/'a<b>c'
$ virsh pool-refresh default
$ virsh vol-dumpxml 'a<b>c' default | head -n2
<volume>
  <name>a<b>c</name>

Oops.  That's not valid XML.  And when we fix the XML
generation, it fails RelaxNG validation.

I'm also tired of seeing <key>(null)</key> in the example
output for volume xml; while we used NULLSTR() to avoid
a NULL deref rather than relying on glibc's printf
extension behavior, it's even better if we avoid the issue
in the first place.  But this requires being careful that
we don't invalidate any storage backends that were relying
on key being unassigned during virStoragVolCreateXML[From].

I would have split this into two patches (one for escaping,
one for avoiding <key>(null)</key>), but since they both
end up touching a lot of the same test files, I ended up
merging it into one.

Note that this patch allows pretty much any volume name
that can appear in a directory (excluding . and .. because
those are special), but does nothing to change the current
(unenforced) RelaxNG claim that pool names will consist
only of letters, numbers, _, -, and +.  Tightening the C
code to match RelaxNG patterns and/or relaxing the grammar
to match the C code for pool names is a task for another
day (but remember, we DID recently tighten C code for
domain names to exclude a leading '.').

* src/conf/storage_conf.c (virStoragePoolSourceFormat)
(virStoragePoolDefFormat, virStorageVolTargetDefFormat)
(virStorageVolDefFormat): Escape user-controlled strings.
(virStorageVolDefParseXML): Parse key, for use in unit tests.
* src/storage/storage_driver.c (storageVolCreateXML)
(storageVolCreateXMLFrom): Ensure parsed key doesn't confuse
volume creation.
* docs/schemas/basictypes.rng (volName): Relax definition.
* tests/storagepoolxml2xmltest.c (mymain): Test it.
* tests/storagevolxml2xmltest.c (mymain): Likewise.
* tests/storagepoolxml2xmlin/pool-dir-naming.xml: New file.
* tests/storagepoolxml2xmlout/pool-dir-naming.xml: Likewise.
* tests/storagevolxml2xmlin/vol-file-naming.xml: Likewise.
* tests/storagevolxml2xmlout/vol-file-naming.xml: Likewise.
* tests/storagevolxml2xmlout/vol-*.xml: Fix fallout.
Signed-off-by: NEric Blake <eblake@redhat.com>

delete extra character 'p' from the comment of virInterfaceCreate
Signed-off-by: NWang Yufei <james.wangyufei@huawei.com>

If a SCSI hostdev is included in an initial domain XML, without a
corresponding controller statement, one is created silently when the
guest is booted.

When hotplugging a SCSI hostdev, a presumption is that the controller
is already present in the domain either from the original XML, or via
an earlier hotplug.

  [root@xxxxxxxx ~]# cat disk.xml
  <hostdev mode='subsystem' type='scsi'>
    <source>
      <adapter name='scsi_host0'/>
      <address bus='0' target='3' unit='1088438288'/>
    </source>
  </hostdev>
  [root@xxxxxxxx ~]# virsh attach-device guest01 disk.xml
  error: Failed to attach device from disk.xml
  error: internal error: unable to execute QEMU command 'device_add': Bus 'scsi0.0' not found

Since the infrastructure is in place, we can also create a controller
silently for use by the hotplugged hostdev device.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>

For systems without a PCI bus, attaching a SCSI controller fails:

  [root@xxxxxxxx ~]# cat controller.xml
  <controller type='scsi' model='virtio-scsi' index='0' />
  [root@xxxxxxxx ~]# virsh attach-device guest01 controller.xml
  error: Failed to attach device from controller.xml
  error: XML error: No PCI buses available

A similar problem occurs with the detach of a controller:

  [root@xxxxxxxx ~]# virsh detach-device guest01 controller.xml
  error: Failed to detach device from controller.xml
  error: operation failed: controller scsi:0 not found

The qemuDomainXXtachPciControllerDevice routines made assumptions
that any caller had a PCI bus.  These routines now selectively calls
PCI functions where necessary, and assigns the device information
type to one appropriate for the bus in use.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

For attach/detach of controller devices, we rename the functions to
remove 'PCI' from their title.  The actual separation of PCI-specific
operations will be handled in the next patch.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>

It makes no sense to go forward to get the parent host number of a
HBA, and treat the HBA as a vHBA with trying to delete it.
Signed-off-by: NOsier Yang <jyang@redhat.com>

These changes allow the correct virtio-blk-device and virtio-net-device
devices to be used for the 'virt' machine type for armv7 rather than the
PCI virtio devices.

A test case was added to qemuxml2argvtest for this change.
Signed-off-by: NClark Laughlin <clark.laughlin@linaro.org>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/network/bridge_driver.c: Consistently use commas.
* src/node_device/node_device_hal.c: Likewise.
* src/node_device/node_device_udev.c: Likewise.
* src/storage/storage_backend_rbd.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/lxc/lxc_container.c: Consistently use commas.
* src/openvz/openvz_driver.c: Likewise.
* src/openvz/openvz_util.c: Likewise.
* src/remote/remote_driver.c: Likewise.
* src/test/test_driver.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/vbox/vbox_tmpl.c: Consistently use commas.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/util/vircommand.c: Consistently use commas.
* src/util/virlog.c: Likewise.
* src/util/virnetdevbandwidth.c: Likewise.
* src/util/virnetdevmacvlan.c: Likewise.
* src/util/virnetdevvportprofile.c: Likewise.
* src/util/virnetlink.c: Likewise.
* src/util/virpci.c: Likewise.
* src/util/virsysinfo.c: Likewise.
* src/util/virusb.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/qemu/qemu_cgroup.c: Consistently use commas.
* src/qemu/qemu_command.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/qemu/qemu_monitor.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/libxl/libxl_driver.c: Consistently use commas.
* src/xen/xend_internal.c: Likewise.
* src/xen/xs_internal.c: Likewise.
* src/xenapi/xenapi_driver.c: Likewise.
* src/xenapi/xenapi_utils.c: Likewise.
* src/xenxs/xen_sxpr.c: Likewise.
* src/xenxs/xen_xm.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/conf/capabilities.c: Consistently use commas.
* src/conf/domain_conf.c: Likewise.
* src/conf/network_conf.c: Likewise.
* src/conf/storage_conf.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/nwfilter/nwfilter_ebiptables_driver.c: Consistently use
commas.
* src/nwfilter/nwfilter_gentech_driver.c: Likewise.
* src/nwfilter/nwfilter_learnipaddr.c: Likewise.
* src/conf/nwfilter_conf.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

For a while we're have random failures of 'securityselinuxtest'
which were not at all reproducible. Fortunately we finally
caught a failure with VIR_TEST_DEBUG=1 enabled. This revealed

TEST: securityselinuxtest
 1) GenLabel "dynamic unconfined, s0, c0.c1023"                       ... OK
 2) GenLabel "dynamic unconfined, s0, c0.c1023"                       ... OK
 3) GenLabel "dynamic unconfined, s0, c0.c1023"                       ... OK
 4) GenLabel "dynamic virtd, s0, c0.c1023"                            ... OK
 5) GenLabel "dynamic virtd, s0, c0.c10"                              ... OK
 6) GenLabel "dynamic virtd, s2-s3, c0.c1023"                         ... OK
 7) GenLabel "dynamic virtd, missing range"                           ... Category two 1024 is out of range 0-1023
FAILED
FAIL: securityselinuxtest

And sure enough we had an off-by-1 in the MCS range code when
the current process has no range set. The test suite randomly
allocates 2 categories from 0->1024 so the chances of hitting
this in the test suite were slim indeed :-)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

All our .pl scripts had the executable bit set, except for one.
Make it consistent (even if we invoke the scripts as an argument
to $(PERL) rather than directly).

* src/check-aclrules.pl: Make executable.
Signed-off-by: NEric Blake <eblake@redhat.com>

Syslog is socket activated since at least systemd v35 so we can drop
this dependency. Debian's linitian otherwise complains about it.

References:

    http://www.freedesktop.org/wiki/Software/systemd/syslog/
    http://lintian.debian.org/tags/systemd-service-file-refers-to-obsolete-target.html