Bug https://bugzilla.redhat.com/show_bug.cgi?id=689374 reported libvirtd
crash during error dispatch.

The reason is that libvirtd uses remoteDispatchConnError() with non-NULL
conn parameter which means that virConnGetLastError() is used instead of
its thread safe replacement virGetLastError().

So when several libvirtd threads are reporting errors at the same time,
the errors can get mixed or corrupted or in case of bad luck libvirtd
itself crashes.

Since Daniel B. is going to rewrite this code from scratch on top of his
RPC infrastructure, I tried to come up with a minimal fix. Thus,
remoteDispatchConnError() now just ignores its conn argument and always
calls virGetLastError(). However, several callers had to be touched as
well, since no libvirt API is allowed to be called before dispatching
the error. Doing so would reset the error and we would have nothing to
dispatch. As a result of that, the code is not very nice but that
doesn't really make daemon/remote.c worse than it is now :-) And it will
all die soon, which is good.

The bug report also contains a reproducer in C which detects both mixed
up error messages and libvirtd crash. Before this patch, I was able to
crash libvirtd in about 20 seconds up to 3 minutes depending on number
of CPU cores (more is better) and luck.

Steps to reproduce this bug:
1. virsh attach-disk domain --source diskimage --target sdb --sourcetype file --driver qemu --subdriver qcow2
   error: Failed to attach disk
   error: operation failed: adding scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 device failed: Property 'scsi-disk.drive' can't find value 'drive-scsi0-0-1'
2. service libvirtd restart
   Stopping libvirtd daemon:                                  [  OK  ]
   Starting libvirtd daemon:                                  [  OK  ]
3. virsh attach-disk domain --source diskimage --target sdb --sourcetype file --driver qemu --subdriver raw
   error: Failed to attach disk
   error: operation failed: adding lsi,id=scsi0,bus=pci.0,addr=0x6 device failed: Duplicate ID 'scsi0' for device

The reason is that we create a new scsi controller but we do not update
/var/run/libvirt/qemu/domain.xml.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

Not every day you see a patch that nukes 27 files!

* .gnulib: Update to latest, for maint.mk improvements
* bootstrap: Resync to gnulib.
* bootstrap.conf (ACLOCAL): Swap the secondary aclocal include
directory, now that bootstrap picks up gnulib/m4 instead of m4.
* Makefile.am (syntax_check_exceptions, EXTRA_DIST): No longer
worry about nuked files.
* cfg.mk (sc_x_sc_dist_check): Delete dead rule.
(VC_LIST_ALWAYS_EXCLUDE_REGEX): Add HACKING.
(exclude_file_name_regexp--sc_*): Inline and simplify contents...
* .x-sc_*: ...from here, then delete the files.

* src/util/command.c (virCommandRunAsync): Since virExec only
creates pidfiles for daemon, enforce this in virCommand.

The ref count was assigned to 1 at creation, then never modified again
until it was decremented just before freeing the object.

* src/conf/domain_conf.h (_virDomainSnapshotObj): Delete unused
field.
(virDomainSnapshotObjUnref): Delete unused prototype.
* src/libvirt_private.syms: Likewise.
* src/conf/domain_conf.c (virDomainSnapshotObjNew)
(virDomainSnapshotObjListDataFree): Update users.
(virDomainSnapshotObjUnref): Delete.

Among others, the missing radvd dependency showed up as:

error: Failed to start network ipv6net
error: Cannot find radvd - Possibly the package isn't installed: No such file
or directory

even when radvd was installed, because the RADVD preprocessor
symbol was missing at configure time.

* libvirt.spec.in (with_network): Add BuildRequires for radvd,
iptables, and ip6tables.
(BuildRequires): Add libxslt and augeas for docs and test.
(with_libvirtd): Add module-init-tools for modprobe.
(with_nwfilter): Add BuildRequires for ebtables.
(with_esx): Fix esx build on RHEL 5, thanks to curl-devel rename.

* libvirt.spec.in (Requires): Reorganize requirements a bit; no
functional change.

Problem:
  "parser.head" is not NULL even if it's free'ed by "virJSONValueFree",
returning "parser.head" when "virJSONValueFromString" fails will cause
unexpected errors (libvirtd will crash sometimes), e.g.
  In function "qemuMonitorJSONArbitraryCommand":

        if (!(cmd = virJSONValueFromString(cmd_str)))
            goto cleanup;

        if (qemuMonitorJSONCommand(mon, cmd, &reply) < 0)
            goto cleanup;

        ......

     cleanup:
        virJSONValueFree(cmd);

  It will continues to send command to monitor even if "virJSONValueFromString"
is failed, and more worse, it trys to free "cmd" again.

  Crash example:
{"error":{"class":"QMPBadInputObject","desc":"Expected 'execute' in QMP input","data":{"expected":"execute"}}}
{"error":{"class":"QMPBadInputObject","desc":"Expected 'execute' in QMP input","data":{"expected":"execute"}}}
error: server closed connection:
error: unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: Connection refused
error: failed to connect to the hypervisor

  This fix is to:
    1) return NULL for failure of "virJSONValueFromString",
    2) and it seems "virJSONValueFree" uses incorrect loop index for type
       of "VIR_JSON_TYPE_OBJECT", fix it together.

* src/util/json.c

Steps to reproduce this bug:
# cat usb.xml
<hostdev mode='subsystem' type='usb'>
  <source>
    <address bus='0x001' device='0x003'/>
  </source>
</hostdev>
# virsh attach-device vm1 usb.xml
error: Failed to attach device from usb.xml
error: server closed connection:

The reason of this bug is that we set data.cgroup to NULL, and this will cause
libvirtd crashed.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

A future patch will change reference counting idioms; consolidating
this pattern now makes the next patch smaller (touch only the new
macro rather than every caller).

* src/qemu/qemu_monitor.c (QEMU_MONITOR_CALLBACK): New helper.
(qemuMonitorGetDiskSecret, qemuMonitorEmitShutdown)
(qemuMonitorEmitReset, qemuMonitorEmitPowerdown)
(qemuMonitorEmitStop, qemuMonitorEmitRTCChange)
(qemuMonitorEmitWatchdog, qemuMonitorEmitIOError)
(qemuMonitorEmitGraphics): Use it to reduce duplication.

Commit cb4aba9b forgot xenapi.

* src/xenapi/xenapi_driver.c (xenapiDriver): Adjust to recent API.

This patch introduces PREASSOCIATE-RR during incoming VM migration on the
destination host. This is similar to the usage of PREASSOCIATE during
migration in 8021qbg libvirt code today. PREASSOCIATE-RR is a VDP operation.
With the latest at IEEE, 8021qbh will need to support VDP operations.
A corresponding enic driver patch to support PREASSOCIATE-RR for 8021qbh
will be posted for net-next-2.6 inclusion soon.

THe veth setup in LXC had a couple of flaws, first brInit did
not report any error when it failed. Second vethCreate() did
not correctly initialize the variable containing the return
code, so could report failure even when it succeeded.

* src/lxc/lxc_driver.c: Report error when brInit fails
* src/lxc/veth.c: Fix uninitialized variable

Enhance the QEMU migration monitoring loop, so that it can get
a signal to change migration speed on the fly

* src/qemu/qemu_domain.h: Add signal for changing speed on the fly
* src/qemu/qemu_driver.c: Wire up virDomainMigrateSetSpeed driver
* src/qemu/qemu_migration.c: Support signal for changing speed

* src/remote/remote_protocol.x: Define wire protocol
* daemon/remote.c, src/remote/remote_driver.c: Add new
  functions for virDomainMigrateSetSpeed API
* src/remote/remote_protocol.c, src/remote/remote_protocol.h,
  daemon/remote_dispatch_args.h, daemon/remote_dispatch_prototypes.h,
  daemon/remote_dispatch_table.h: Re-generate files

It is possible to set a migration speed limit when starting
migration. This new API allows the speed limit to be changed
on the fly to adjust to changing conditions

* src/driver.h, src/libvirt.c, src/libvirt_public.syms,
  include/libvirt/libvirt.h.in: Add virDomainMigrateSetMaxSpeed
* src/esx/esx_driver.c, src/lxc/lxc_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/vmware/vmware_driver.c, src/xen/xen_driver.c,
  src/libxl/libxl_driver.c: Stub new API

* AUTHORS: Update, to satisfy 'make syntax-check'.

The xen RPM in Fedora isn't new enough to support libxl builds
yet. Disable it until Fedora 16

* libvirt.spec.in: Disable libxl on Fedora < 16

fallback to HMP drive_add/drive_del commands if not found in QMP

Avoids calling text monitor methods when it is know they will not
succeed and also results in nicer error messages.

Fix for bug https://bugzilla.redhat.com/show_bug.cgi?id=618970

The "prepare" hook is called very early in the VM statup process
before device labeling, so that it can allocate ressources not
managed by libvirt, such as DRBD, or for instance create missing
bridges and vlan interfaces.
* src/util/hooks.c src/util/hooks.h: add definitions for new hooks
  VIR_HOOK_QEMU_OP_PREPARE and VIR_HOOK_QEMU_OP_RELEASE
* src/qemu/qemu_process.c: use them in qemuProcessStart and
  qemuProcessStop()

Wen Congyang gained commiter access

Created a new section of previous commiters to the project
but not involved much anymore: Karel Zak, Atsushi SAKAI,
Dave Leskovec and Dan Smith

With only a single caller to these two monitor commands, I
didn't need to wrap a new WithFds version, but just change
the command itself.

* src/qemu/qemu_monitor.h (qemuMonitorAddNetdev)
(qemuMonitorAddHostNetwork): Add parameters.
* src/qemu/qemu_monitor.c (qemuMonitorAddNetdev)
(qemuMonitorAddHostNetwork): Add support for fd passing.
* src/qemu/qemu_hotplug.c (qemuDomainAttachNetDevice): Use it to
simplify code.

This is also a bug fix - on the error path, qemu_hotplug would
leave the configfd file leaked into qemu.  At least the next
attempt to hotplug a PCI device would reuse the same fdname,
and when the qemu getfd monitor command gets a new fd by the
same name as an earlier one, it closes the earlier one, so there
is no risk of qemu running out of fds.

* src/qemu/qemu_monitor.h (qemuMonitorAddDeviceWithFd): New
prototype.
* src/qemu/qemu_monitor.c (qemuMonitorAddDevice): Move guts...
(qemuMonitorAddDeviceWithFd): ...to new function, and add support
for fd passing.
* src/qemu/qemu_hotplug.c (qemuDomainAttachHostPciDevice): Use it
to simplify code.
Suggested by Daniel P. Berrange.

qemu_monitor was already returning -1 and setting errno to EINVAL
on any attempt to send an fd without a unix socket, but this was
a silent failure in the case of qemuDomainAttachHostPciDevice.
Meanwhile, qemuDomainAttachNetDevice was doing some sanity checking
for a better error message; it's better to consolidate that to a
central point in the API.

* src/qemu/qemu_hotplug.c (qemuDomainAttachNetDevice): Move sanity
checking...
* src/qemu/qemu_monitor.c (qemuMonitorSendFileHandle): ...into
central location.
Suggested by Chris Wright.

https://bugzilla.redhat.com/show_bug.cgi?id=684655 points out
a regression introduced in commit 2215050e - non-root users
can't connect to qemu:///session because libvirtd dies when
it can't use pciaccess initialization.

* src/node_device/node_device_udev.c (udevDeviceMonitorStartup):
Don't abort udev driver (and libvirtd overall) if non-root user
can't use pciaccess.

Valgrind caught that our log wrap-around was going 1 past the end.
Regression introduced in commit b16f47ab; previously the
buffer was static and size+1 bytes, but now it is dynamic and
exactly size bytes.

* src/util/logging.c (virLogStr): Don't write past end of log.

We have reported error in the function prepareCall(), and
the error is not only OOM error. So we should not report
OOM error in the function call() when prepareCall() failed.

If virFileIsExecutable is to replace access(file,X_OK), then
errno must be usable on failure.

* src/util/util.c (virFileIsExecutable): Set errno on failure.

<imagelable> is not generated by running domain, actually we parse
it in src/conf/domain_conf.c, this patch is to fix it, otherwise any
validation (virt-xml-validate) on the domain xml dumped from shutoff
domain containing <imagelable> will fail.

* docs/schemas/domain.rng

The current description suggests that you always have to provide
a valid typeVer pointer. But if you want only the libvirt version
it's also possible to set type and typeVer to NULL to skip the
hypervisor part.

This patch enables cgroup controllers as much as possible by skipping
the creation of blkio controller when running with old kernels that
doesn't support multi-level directory for blkio controller.
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The libxenlight driver does not build against the tech preview
version of libxenlight in Xen 4.0.  Only enable building the
driver against more complete libxenlight found in Xen 4.1.

THREADS.txt states that the contents of vm should not be read or
modified while the vm lock is not held, but that the lock must not
be held while performing a monitor command.  This fixes all the
offenders that I could find.

* src/qemu/qemu_process.c (qemuProcessStartCPUs)
(qemuProcessInitPasswords, qemuProcessStart): Don't modify or
refer to vm state outside lock.
* src/qemu/qemu_driver.c (qemudDomainHotplugVcpus): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphicsPasswords):
Likewise.

This is detailed in:

  https://bugzilla.redhat.com/show_bug.cgi?id=688957

Since radvd is executed by daemonizing it, the attempt to exec the
radvd binary doesn't happen until after libvirtd has already received
an exit code from the intermediate forked process, so no error is
detected or logged by __virExec().

We can't require radvd as a prerequisite for the libvirt package (many
installations don't use IPv6, so they don't need it), so instead we
add in a check to verify there is an executable radvd binary prior to
trying to exec it.

This patch adds missing curly brackets to an if
statement in openvzListDefinedDomains()

* po/POTFILES.in: Add src/util/hash.c for 'make syntax-check'.

When SASL is active, it was possible that we read and decoded
more data off the wire than we initially wanted. The loop
processing this data terminated after only one message to
avoid delaying the calling thread, but this could delay
event delivery. As long as there is decoded SASL data in
memory, we must process it, before returning to the poll()
event loop.

This is a counterpart to the same kind of issue solved in

  commit 68d2c348

in a different area of the code

* src/remote/remote_driver.c: Process all pending SASL data