If an LXC VM fails to start, quite a few cleanup paths will
result in the original error message being overwritten. Some
other cleanup paths also forgot to actually terminate the VM.

* src/lxc/lxc_driver.c: Ensure VM is terminated on startup
  failure and preserve original error

The LXC code for mounting container filesystems from block devices
tries all filesystems in /etc/filesystems and possibly those in
/proc/filesystems. The regular mount binary, however, first tries
using libblkid to detect the format. Add support for doing the same
in libvirt, since Fedora's /etc/filesystems is missing many formats,
most notably ext4 which is the default filesystem Fedora uses!

* src/Makefile.am: Link libvirt_lxc to libblkid
* src/lxc/lxc_container.c: Probe filesystem format with libblkid

If we looped through /etc/filesystems trying to mount with each
type and failed all options, we forget to actually raise an
error message.

* src/lxc/lxc_container.c: Raise error if unable to detect
  the filesystems. Also fix existing error message

The kernel automounter is mostly broken wrt to containers. Most
notably if you start a new filesystem namespace and then attempt
to unmount any autofs filesystem, it will typically fail with a
weird error message like

  Failed to unmount '/.oldroot/sys/kernel/security':Too many levels of symbolic links

Attempting to detach the autofs mount using umount2(MNT_DETACH)
will also fail with the same error. Therefore if we get any error on
unmount()ing a filesystem from the old root FS when starting a
container, we must immediately break out and detach the entire
old root filesystem (ignoring any mounts below it).

This has the effect of making the old root filesystem inaccessible
to anything inside the container, but at the cost that the mounts
live on in the kernel until the container exits. Given that SystemD
uses autofs by default, we need LXC to be robust this scenario and
thus this tradeoff is worthwhile.

* src/lxc/lxc_container.c: Detach root filesystem if any umount
  operation fails.

The /etc/filesystems file can contain a '*' on the last line to
indicate that /proc/filessystems should be tried next. We have
a check that this '*' only occurs on the last line. Unfortunately
when we then start reading /proc/filesystems, we mistakenly think
we've seen '*' in /proc/filesystems and fail

* src/lxc/lxc_container.c: Skip '*' validation when we're reading
  /proc/filesystems

Only some of the return paths of lxcContainerWaitForContinue will
have set errno. In other paths we need to set it manually to avoid
the caller getting a random stale errno value

* src/lxc/lxc_container.c: Set errno in lxcContainerWaitForContinue

We already have a /var/lib/libvirt/images for OS install images.
We need a separate /var/lib/libvirt/filesystems for OS install
trees, since SELinux labelling will be different

* libvirt.spec.in: Add /var/lib/libvirt/filesystems
* src/Makefile.am: Create /var/lib/libvirt/filesystems

Allow the datacenter and compute resource parts of the path
to be prefixed with folders. Therefore, the way the path is
parsed has changed. Before, it was split in 2 or 3 items and
the items' meanings were determined by their positions. Now
the path can have 2 or more items and the the vCenter server
is asked whether a folder, datacenter of compute resource
with the specified name exists at the current hierarchy level.

Before the datacenter and compute resource lookup automatically
traversed folders during lookup. This is logic got removed
and folders have to be specified explicitly.

The proper datacenter path including folders is now used when
accessing a datastore over HTTPS. This makes virsh dumpxml
and define work for datacenters in folders.

https://bugzilla.redhat.com/show_bug.cgi?id=732676

And virtual hardware version 8.

with /etc/libvirt/libvirt.conf below:
uri_aliases = [
  "hail=qemu:///system",
  "sleet=qemu+ssh://root 9 115 122 57/system",
  "sam=qemu+unix:///system?socket=/var/run/libvirt/libvirt-sock",
]
Neither "virsh -c hailly" nor "hai" should result in matching "hail=qemu:///system"

Fix URI alias prefix matching when connecting
Signed-off-by: NWen Ruo Lv <lvroyce@linux.vnet.ibm.com>

* docs/formatdomain.html.in: Use dev, not def.
Reported by Alexander Biryukov.

If daemon is using SASL it reads client data into a cache. This cache is
big (usually 65KB) and can thus contain 2 or more messages. However,
on socket event we can dispatch only one message. So if we read two
messages at once, the second will not be dispatched as the socket event
goes away with filling the cache.
Moreover, when dispatching the cache we need to remember to take care
of client max requests limit.

If we are comparing storage pools we must skip comparing with
ourself, so that re-defining an existing pool works

* conf/storage_conf.c: Skip self when comparing

The qemu RBD driver needs access to the conn in order to get the secret
needed for connecting to the ceph cluster.
Signed-off-by: NSage Weil <sage@newdream.net>

Detected by Coverity. Leak introduced in commit 6cabc0b0.

* src/qemu/qemu_command.c: Clean up on failure.
Signed-off-by: NAlex Jia <ajia@redhat.com>

This was fixed in be757a3f for libvirt.c.

This patch does some cleanups to my previous startupPolicy patchset.

To support "managed" mode of host PCI device, we record the original
states (unbind_from_stub, remove_slot, and reprobe) so that could
reattach the device to host with original driver. But there is no XML
for theses attrs, and thus after daemon is restarted, we lose the
original states. It's easy to reproduce:

    1) virsh start domain
    2) virsh attach-device dom hostpci.xml (in 'managed' mode)
    3) service libvirtd restart
    4) virsh destroy domain

    You will see the device won't be bound to the original driver
if there was one.

This patch is to solve the problem by introducing internal XML
(won't be dumped to user, only dumped to status XML). The XML is:
    <origstates>
      <unbind/>
      <remove_slot/>
      <reprobe/>
    </origstates>

Which will be child node of <hostdev><source>...</souce></hostdev>.
(only for PCI device).

A new struct "virDomainHostdevOrigStates" is introduced for the XML,
and the according members are updated when preparing the PCI device.
And function "qemuUpdateActivePciHostdevs" is modified to honor
the original states. Use of qemuGetPciHostDeviceList is removed
in function "qemuUpdateActivePciHostdevs", and the "managed" value of
the device config is honored by the change. This fixes another problem
alongside:

    qemuGetPciHostDeviceList set the device as "managed" force
    regardless of whether the device is configured as "managed='yes'"
    or not in XML, which is not right.

Deal with the incompatible changes in the VirtualBox 4.1 API.

INetworkAdapter has its different AttachTo* method replaced by
a settable attachmentType property.

The maximum number of network adapters is now requestable per
chipset type.

The OpenMedium method got a bool parameter to request opening
a medium under a new IID.

Shared folders are handled as filesystems and can also be hotplugged.

privP->session->error_description is a list and in order to get the
complete error message all parts of the list should be concatenated.
xenapiSessionErrorHandler does this when its third parameter is NULL.
The current code discards all but the first part of the error message
resulting in a potentially incomplete error message.

This partly reverts 006be75e, that tried to avoid reporting
a (null) in the error message. The actual problem is more general in
returnErrorFromSession that might return NULL if there is no error.

Make sure that returnErrorFromSession return non-NULL always. Also
don't skip the last error message part.

- changed some return 1's to return -1
- changed if (rc) error checks to if (rc < 0)
- fixed some other minor convention violations

I might have missed some. Can fix in another patch or can respin
Signed-off-by: NRoopa Prabhu <roprabhu@cisco.com>
Reported-by: NEric Blake <eblake@redhat.com>
Reported-by: NLaine Stump <laine@laine.org>
Signed-off-by: NEric Blake <eblake@redhat.com>

The types used in domaincommon.rng and secret.rng should be the same.

Move genericName to basictypes.rng, then drop redundant types now
that secret.rng uses basictypes.rng.
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

When using the xml as below:
------------------------------------------------------
<devices>
  <emulator>/home/soulxu/data/work-code/qemu-kvm/x86_64-softmmu/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source file='/home/soulxu/data/VM/images/linux.img'/>
    <target dev='vda' bus='virtio'/>
    <address type='drive' controller='0' bus='0' unit='0'/>
  </disk>
  <input type='mouse' bus='ps2'/>
  <graphics type='vnc' port='-1' autoport='yes'/>
  <video>
    <model type='cirrus' vram='9216' heads='1'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
  </video>
  <memballoon model='virtio'>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
  </memballoon>
</devices>
------------------------------------------------------

Then can't startup qemu, the error message as below:
virsh # start test-vm
error: Failed to start domain test-vm
error: internal error process exited while connecting to monitor: qemu-system-x86_64: -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3: PCI: slot 3 function 0 not available for virtio-balloon-pci, in use by virtio-blk-pci
qemu-system-x86_64: -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3: Device 'virtio-balloon-pci' could not be initialized

So adding check for bus type and address type. Only the address of pci type support by virtio bus.
Signed-off-by: NXu He Jie <xuhj@linux.vnet.ibm.com>

In hand-applying Josh and Sage's patch, I missed out on a break.

* src/conf/secret_conf.c (virSecretDefFree): Fix my botch.

Add additional fields to let you specify the how to authenticate with a disk.
The secret to use may be referenced by a usage string or a UUID, i.e.:

<auth username='myuser'>
 <secret type='ceph' usage='secretname'/>
</auth>

or

<auth username='myuser'>
 <secret type='ceph' uuid='0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f'/>
</auth>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>

Add a new secret type to store a Ceph authentication key. The name
is simply an identifier for easy human reference.

The xml looks like this:

<secret ephemeral='no' private='no'>
 <uuid>0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f</uuid>
 <usage type='ceph'>
   <name>mycluster_admin</name>
 </usage>
</secret>
Signed-off-by: NSage Weil <sage@newdream.net>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.net>

Leak introduced in commit c1bc3d89.
Detected by valgrind:

==18462== 1,100 bytes in 1 blocks are definitely lost in loss record 183 of 184
==18462==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==18462==    by 0x4A06167: realloc (vg_replace_malloc.c:525)
==18462==    by 0x4AADBB: virReallocN (memory.c:161)
==18462==    by 0x4A975E: virBufferGrow (buf.c:117)
==18462==    by 0x4A9D92: virBufferVasprintf (buf.c:290)
==18462==    by 0x4A9EF7: virBufferAsprintf (buf.c:263)
==18462==    by 0x429488: qemuBuildControllerDevStr (qemu_command.c:1993)
==18462==    by 0x42C4B6: qemuBuildCommandLine (qemu_command.c:3803)
==18462==    by 0x41A604: testCompareXMLToArgvHelper (qemuxml2argvtest.c:124)
==18462==    by 0x41BB81: virtTestRun (testutils.c:141)
==18462==    by 0x416DFF: mymain (qemuxml2argvtest.c:369)
==18462==    by 0x41B277: virtTestMain (testutils.c:696)
==18462==
==18462== LEAK SUMMARY:
==18462==    definitely lost: 1,100 bytes in 1 blocks
==18462==    indirectly lost: 0 bytes in 0 blocks

* src/qemu/qemu_command.c (qemuBuildCommandLine): Clean up on success.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Detected by Coverity.  The fix in 2c27dfae didn't catch all bad
instances of memcpy().  Thankfully, on further analysis, all of
the problematic uses are only triggered by old qemu that lacks
-device.

* src/qemu/qemu_hotplug.c (qemuDomainAttachPciDiskDevice)
(qemuDomainAttachNetDevice, qemuDomainAttachHostPciDevice): Init
all fields since monitor only populates some of them.

It forgets to move a pointer to a buffer for UUID and as a result
fills only the first byte of the buffer.

Since it needs to access file descriptors passed in the msg,
the RPC driver for virDomainOpenGraphics needs to be manually
implemented.

* daemon/remote.c: RPC server dispatcher
* src/remote/remote_driver.c: RPC client dispatcher
* src/remote/remote_protocol.x: Define protocol

The RPC server classes are extended to allow FDs to be received
from clients with calls. There is not currently any way for a
procedure to pass FDs back to the client with replies

* daemon/remote.c, src/rpc/gendispatch.pl: Change virNetMessageHeaderPtr
  param to virNetMessagePtr in dispatcher impls
* src/rpc/virnetserver.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c, src/rpc/virnetserverprogram.h:
  Extend to support FD passing

Extend the RPC client code to allow file descriptors to be sent
to the server with calls, and received back with replies.

* src/remote/remote_driver.c: Stub extra args
* src/libvirt_private.syms, src/rpc/virnetclient.c,
  src/rpc/virnetclient.h, src/rpc/virnetclientprogram.c,
  src/rpc/virnetclientprogram.h: Extend APIs to allow
  FD passing

Define two new RPC message types VIR_NET_CALL_WITH_FDS and
VIR_NET_REPLY_WITH_FDS. These message types are equivalent
to VIR_NET_CALL and VIR_NET_REPLY, except that between the
message header, and payload there is a 32-bit integer field
specifying how many file descriptors have been passed.

The actual file descriptors are sent/recv'd out of band.

* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h,
  src/libvirt_private.syms: Add support for handling
  passed file descriptors
* src/rpc/virnetprotocol.x: Extend protocol for FD
  passing

Add APIs to the virNetSocket object, to allow file descriptors
to be sent/received over UNIX domain socket connections

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h,
  src/libvirt_private.syms: Add APIs for FD send/recv

The QEMU monitor command 'add_client' can be used to connect to
a VNC or SPICE graphics display. This allows for implementation
of the virDomainOpenGraphics API

* src/qemu/qemu_driver.c: Implement virDomainOpenGraphics
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Add binding for 'add_client' command

Not all VNC/SPICE servers use a TCP socket for their connections.
It is possible to configure a UNIX socket server. The graphics
event must thus include a UNIX socket address type.

* include/libvirt/libvirt.h.in: Add UNIX socket address type
  for graphics event
* src/qemu/qemu_monitor_json.c: Add 'unix' string to address
  type enum

The virDomainOpenGraphics API allows a libvirt client to pass in
a file descriptor for an open socket pair, and get it connected
to the graphics display of the guest. This is limited to working
with local libvirt hypervisors connected over a UNIX domain
socket, since it will use UNIX FD passing

* include/libvirt/libvirt.h.in: Define virDomainOpenGraphics
* src/driver.h: Define driver for virDomainOpenGraphics
* src/libvirt_public.syms, src/libvirt.c: Entry point for
  virDomainOpenGraphics
* src/libvirt_internal.h: VIR_DRV_FEATURE_FD_PASSING

The 5th line of every xfig file has 'Letter   ' which
annoys GITs trailing-whitespace commit hook. Hand edit
the files to remove that trailing whitespace

* docs/*.fig: Remove trailing whitespace

This adds a page documenting many aspects of migration:

 - The types of migration (managed direct, p2p, unmanaged direct)
 - Data transports (native, tunnelled)
 - Migration URIs
 - Config file handling
 - Example scenarios

* libvirt.css: Rules for data tables and diagrams
* Makefile.am: Include extra png/fig files
* migration-managed-direct.fig, migration-managed-direct.png,
  migration-managed-direct.png, migration-managed-p2p.png,
  migration-native.fig, migration-native.png,
  migration-tunnel.fig, migration-tunnel.png,
  migration-unmanaged-direct.fig, migration-unmanaged-direct.png:
  Diagrams of migration
* migration.html.in, sitemap.html.in: New migration doc