In hand-applying Josh and Sage's patch, I missed out on a break.

* src/conf/secret_conf.c (virSecretDefFree): Fix my botch.

Add additional fields to let you specify the how to authenticate with a disk.
The secret to use may be referenced by a usage string or a UUID, i.e.:

<auth username='myuser'>
 <secret type='ceph' usage='secretname'/>
</auth>

or

<auth username='myuser'>
 <secret type='ceph' uuid='0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f'/>
</auth>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>

Add a new secret type to store a Ceph authentication key. The name
is simply an identifier for easy human reference.

The xml looks like this:

<secret ephemeral='no' private='no'>
 <uuid>0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f</uuid>
 <usage type='ceph'>
   <name>mycluster_admin</name>
 </usage>
</secret>
Signed-off-by: NSage Weil <sage@newdream.net>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.net>

Leak introduced in commit c1bc3d89.
Detected by valgrind:

==18462== 1,100 bytes in 1 blocks are definitely lost in loss record 183 of 184
==18462==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==18462==    by 0x4A06167: realloc (vg_replace_malloc.c:525)
==18462==    by 0x4AADBB: virReallocN (memory.c:161)
==18462==    by 0x4A975E: virBufferGrow (buf.c:117)
==18462==    by 0x4A9D92: virBufferVasprintf (buf.c:290)
==18462==    by 0x4A9EF7: virBufferAsprintf (buf.c:263)
==18462==    by 0x429488: qemuBuildControllerDevStr (qemu_command.c:1993)
==18462==    by 0x42C4B6: qemuBuildCommandLine (qemu_command.c:3803)
==18462==    by 0x41A604: testCompareXMLToArgvHelper (qemuxml2argvtest.c:124)
==18462==    by 0x41BB81: virtTestRun (testutils.c:141)
==18462==    by 0x416DFF: mymain (qemuxml2argvtest.c:369)
==18462==    by 0x41B277: virtTestMain (testutils.c:696)
==18462==
==18462== LEAK SUMMARY:
==18462==    definitely lost: 1,100 bytes in 1 blocks
==18462==    indirectly lost: 0 bytes in 0 blocks

* src/qemu/qemu_command.c (qemuBuildCommandLine): Clean up on success.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Detected by Coverity.  The fix in 2c27dfae didn't catch all bad
instances of memcpy().  Thankfully, on further analysis, all of
the problematic uses are only triggered by old qemu that lacks
-device.

* src/qemu/qemu_hotplug.c (qemuDomainAttachPciDiskDevice)
(qemuDomainAttachNetDevice, qemuDomainAttachHostPciDevice): Init
all fields since monitor only populates some of them.

It forgets to move a pointer to a buffer for UUID and as a result
fills only the first byte of the buffer.

Since it needs to access file descriptors passed in the msg,
the RPC driver for virDomainOpenGraphics needs to be manually
implemented.

* daemon/remote.c: RPC server dispatcher
* src/remote/remote_driver.c: RPC client dispatcher
* src/remote/remote_protocol.x: Define protocol

The RPC server classes are extended to allow FDs to be received
from clients with calls. There is not currently any way for a
procedure to pass FDs back to the client with replies

* daemon/remote.c, src/rpc/gendispatch.pl: Change virNetMessageHeaderPtr
  param to virNetMessagePtr in dispatcher impls
* src/rpc/virnetserver.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c, src/rpc/virnetserverprogram.h:
  Extend to support FD passing

Extend the RPC client code to allow file descriptors to be sent
to the server with calls, and received back with replies.

* src/remote/remote_driver.c: Stub extra args
* src/libvirt_private.syms, src/rpc/virnetclient.c,
  src/rpc/virnetclient.h, src/rpc/virnetclientprogram.c,
  src/rpc/virnetclientprogram.h: Extend APIs to allow
  FD passing

Define two new RPC message types VIR_NET_CALL_WITH_FDS and
VIR_NET_REPLY_WITH_FDS. These message types are equivalent
to VIR_NET_CALL and VIR_NET_REPLY, except that between the
message header, and payload there is a 32-bit integer field
specifying how many file descriptors have been passed.

The actual file descriptors are sent/recv'd out of band.

* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h,
  src/libvirt_private.syms: Add support for handling
  passed file descriptors
* src/rpc/virnetprotocol.x: Extend protocol for FD
  passing

Add APIs to the virNetSocket object, to allow file descriptors
to be sent/received over UNIX domain socket connections

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h,
  src/libvirt_private.syms: Add APIs for FD send/recv

The QEMU monitor command 'add_client' can be used to connect to
a VNC or SPICE graphics display. This allows for implementation
of the virDomainOpenGraphics API

* src/qemu/qemu_driver.c: Implement virDomainOpenGraphics
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Add binding for 'add_client' command

Not all VNC/SPICE servers use a TCP socket for their connections.
It is possible to configure a UNIX socket server. The graphics
event must thus include a UNIX socket address type.

* include/libvirt/libvirt.h.in: Add UNIX socket address type
  for graphics event
* src/qemu/qemu_monitor_json.c: Add 'unix' string to address
  type enum

The virDomainOpenGraphics API allows a libvirt client to pass in
a file descriptor for an open socket pair, and get it connected
to the graphics display of the guest. This is limited to working
with local libvirt hypervisors connected over a UNIX domain
socket, since it will use UNIX FD passing

* include/libvirt/libvirt.h.in: Define virDomainOpenGraphics
* src/driver.h: Define driver for virDomainOpenGraphics
* src/libvirt_public.syms, src/libvirt.c: Entry point for
  virDomainOpenGraphics
* src/libvirt_internal.h: VIR_DRV_FEATURE_FD_PASSING

The 5th line of every xfig file has 'Letter   ' which
annoys GITs trailing-whitespace commit hook. Hand edit
the files to remove that trailing whitespace

* docs/*.fig: Remove trailing whitespace

This adds a page documenting many aspects of migration:

 - The types of migration (managed direct, p2p, unmanaged direct)
 - Data transports (native, tunnelled)
 - Migration URIs
 - Config file handling
 - Example scenarios

* libvirt.css: Rules for data tables and diagrams
* Makefile.am: Include extra png/fig files
* migration-managed-direct.fig, migration-managed-direct.png,
  migration-managed-direct.png, migration-managed-p2p.png,
  migration-native.fig, migration-native.png,
  migration-tunnel.fig, migration-tunnel.png,
  migration-unmanaged-direct.fig, migration-unmanaged-direct.png:
  Diagrams of migration
* migration.html.in, sitemap.html.in: New migration doc

If vol-create-from is failed due to 'input volume not found',
virsh outputs like this:
  $ sudo virsh vol-create-from testpool test-vol.xml test.img
  error: failed to get vol 'test.img', specifying --pool might help
  error: Storage volume not found: no storage vol with matching path
However, '--pool' is incorrect because it is already specified as
second argument ('testpool' in this case). It should be "--inputpool".

The patch fixes this by using pooloptname, which will be "inputpool"
in this case and "pool" in other cases, as error message.

This refactors the TAP creation code out of brAddTap into a new
function brCreateTap to allow it to be used on its own. I have also
changed ifSetInterfaceMac to brSetInterfaceMac and exported it since
it is will be needed by code outside of util/bridge.c in the next
patch.

 AUTHORS                 |    1 +
 src/libvirt_bridge.syms |    2 +
 src/util/bridge.c       |  116 +++++++++++++++++++++++++++++++----------------
 src/util/bridge.h       |    9 ++++
 4 files changed, 89 insertions(+), 39 deletions(-)

Every time we write XML into a file we call virEmitXMLWarning to write a
warning that the file is automatically generated. virXMLSaveFile
simplifies this into a single step and makes rewriting existing XML file
safe by using virFileRewrite internally.

When saving config files we just overwrite old content of the file. In
case something fails during that process (e.g. disk gets full) we lose
both old and new content. This patch makes the process more robust by
writing the new content into a separate file and only if that succeeds
the original file is atomically replaced with the new one.

This change adds some systemtap/dtrace probes to the QEMU monitor
client code. In particular it allows watching of all operations
for a VM

* examples/systemtap/qemu-monitor.stp: Watch all monitor commands
* src/Makefile.am: Passing libdir/bindir/sbindir to dtrace2systemtap.pl
* src/dtrace2systemtap.pl: Accept libdir/bindir/sbindir as args
  and look for '# binary:' comment to mark probes against libvirtd
  vs libvirt.so
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor_json.c,
  src/qemu/qemu_monitor_text.c: Add probes for key functions

Previous commit clears number of items alocated in lxcSetupLoopDevices
if VIR_REALLOC_N fails. In that case, the pointer is not NULL, and
causes leaking FDs that have been allocated.

 *  src/lxc/lxc_controller.c: revert zeroing array size

* src/util/util.c: s/git_t/gid_t/ in parameter list of virFileAccessibleAs

If the function lxcSetupLoopDevices(def, &nloopDevs, &loopDevs) failed,
the variable loopDevs will keep a initial NULL value, however, the
function VIR_FORCE_CLOSE(loopDevs[i]) will directly deref it.

This patch also fixes returning a bogous number of devices from
lxcSetupLoopDevices on an error path.

* rc/lxc/lxc_controller.c: fixed a null pointer dereference.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Cppcheck detected a syntaxError on lxcDomainInterfaceStats.

* src/lxc/lxc_driver.c: fixed missing '{' in the function lxcDomainInterfaceStats.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Rather than making all clients of monitor commands that are JSON-only
check whether yajl support was compiled in, it is simpler to just
avoid setting the capability bit up front if we can't use the capability.

* src/qemu/qemu_capabilities.c (qemuCapsComputeCmdFlags): Only set
capability bit if we also have yajl library to use it.
* src/qemu/qemu_driver.c (qemuDomainReboot): Drop #ifdefs.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* tests/qemuhelptest.c (testHelpStrParsing): Pass test even
without yajl.
* tests/qemuxml2argvtest.c (mymain): Simplify use of json flag.
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-error-*.args:
Update expected results to match.

Break some long lines, and use more efficient functions when possible,
such as relying on virBufferEscapeString to skip output on a NULL arg.
Ensure that output does not embed newlines, since auto-indent won't
work in those situations.

* src/conf/domain_conf.c (virDomainTimerDefFormat): Break output lines.
(virDomainDefFormatInternal, virDomainDiskDefFormat)
(virDomainActualNetDefFormat, virDomainNetDefFormat)
(virDomainHostdevDefFormat): Minor cleanups.

Use auto-indent in more places.

* src/conf/storage_encryption_conf.h (virStorageEncryptionFormat):
Drop parameter.
* src/conf/storage_encryption_conf.c (virStorageEncryptionFormat)
(virStorageEncryptionSecretFormat): Simplify with auto-indent.
* src/conf/domain_conf.c (virDomainDiskDefFormat): Adjust caller.
* src/conf/storage_conf.c (virStorageVolTargetDefFormat): Likewise.

Fixing this involved some refactoring of common code out of
domain_conf and nwfilter_conf into nwfilter_params.

* src/conf/nwfilter_params.h (virNWFilterFormatParamAttributes):
Adjust signature.
* src/conf/nwfilter_params.c (_formatParameterAttrs)
(virNWFilterFormatParamAttributes): Adjust indentation handling,
and handle filterref here.
(formatterParam): Delete unused struct.
* src/conf/domain_conf.c (virDomainNetDefFormat): Adjust caller.
* src/conf/nwfilter_conf.c (virNWFilterIncludeDefFormat): Likewise.

Detected by Coverity.  Only possible if qemu-img gives bogus output,
but we might as well be robust.

* src/storage/storage_backend.c
(virStorageBackendQEMUImgBackingFormat): Check for strstr failure.

Per the discussion here, LXC on RHEL 5 makes no sense.
https://www.redhat.com/archives/libvir-list/2011-September/msg01169.html

* configure.ac (with_lxc): Reject RHEL 5.x LXC as too old.

Commit 1726a736 hacked around MacOS' lack of fdatasync, since
gnulib did not have it at the time.  But now that we use newer
gnulib, we can avoid the hack.

* bootstrap.conf (gnulib_modules): Add fdatasync.
* configure.ac (AC_CHECK_FUNCS_ONCE): Drop our own check.

* src/remote_protocol-structs: Add new event

Extend the nwfilter schema to support the continue and return targets.

Commit 458b7099 introduced this
function. However it was not added into libvirt_private.syms so
the compilation may not succeed on some hosts.

If a disk source gets dropped because it is not accessible,
mgmt application might want to be informed about this. Therefore
we need to emit an event. The event presented in this patch
is however a bit superset of what written above. The reason is simple:
an intention to be easily expanded, e.g. on 'user ejected disk
in guest' events. Therefore, callback gets source string and disk alias
(which should be unique among a domain) and reason (an integer);

This patch implements on_missing feature in qemu driver.
Upon qemu startup process an accessibility of CDROMs
and floppy disks is checked. The source might get dropped
if unavailable and on_missing is set accordingly.
No event is emit thought. Look for follow up patch.

This patch is rather cosmetic as it only moves device alias
assignation from command line construction just before that.
However, it is needed in connotation of previous and next patch.

This function checks if a given path is accessible under
given uid and gid.