The commit that prevents disk corruption on domain shutdown
(96fc4784) causes regression with QEMU
0.14.* and 0.15.* because of a regression bug in QEMU that was fixed
only recently in QEMU git. The affected versions of QEMU do not quit on
SIGTERM if started with -no-shutdown, which we use to implement fake
reboot. Since -no-shutdown tells QEMU not to quit automatically on guest
shutdown, domains started using the affected QEMU cannot be shutdown
properly and stay in a paused state.

This patch disables fake reboot feature on such QEMU by not using
-no-shutdown, which makes shutdown work as expected. However,
virDomainReboot will not work in this case and it will report "Requested
operation is not valid: Reboot is not supported with this QEMU binary".

Bug introduced in commit 675464b1.  On an OOM, this would try to
dereference a char* and free the contents as a pointer, which is
doomed to failure.

Adding a syntax check will prevent mistakes like this in the future.

* cfg.mk (sc_prohibit_internal_functions): New syntax check.
(exclude_file_name_regexp--sc_prohibit_internal_functions): Add
exemptions.
* daemon/remote.c (remoteRelayDomainEventIOError)
(remoteRelayDomainEventIOErrorReason)
(remoteRelayDomainEventGraphics, remoteRelayDomainEventBlockJob):
Use correct free function.

The next patch will add a syntax check that flags this usage in xen
as awkward - while it was valid memory management, it was very hard
to maintain.  Swapping to a more traditional allocation may be a bit
slower, but easier to understand.

* src/xen/xend_internal.c (xenDaemonListDomainsOld): Use two-level
allocation, rather than abusing allocation function.
(xenDaemonLookupByUUID): Update caller.

Variable "name" is never used in the codes, it's useless.

gcc warns when building libvirt 0.9.5 on a 32-bit machine:

qemu/qemu_migration.c: In function 'qemuMigrationToFile':
qemu/qemu_migration.c:2727:38: error: large integer implicitly truncated to unsigned type [-Woverflow]

* src/qemu/qemu_domain.h (QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX): Cap
to long when building for 32-bit platform.

* configure.ac docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: fetch updated translations from transifex and rebuilt

remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError,
remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics
were using const string directly in rpc structure, before calling
remoteDispatchDomainEventSend(). But that routine now frees up all
the pointed allocated memory from the rpc structure and we end up
with a double free.
This now strdup() all the strings passed and provide mem_error goto
labels to be used when an allocation error occurs.
Note that the cleanup isn't completely finished because all relaying
function also call make_nonnull_domain() which also allocate a string
and never handle the error case. This patches doesn't try to address
this as this is only error correctness a priori and touches far more
functions in this module:

* daemon/remote.c: fix string allocations and memory error handling
  for remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError,
  remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics

Inexplicably the sanlock code all got placed under the GPLv2-only,
so libvirt's use of sanlock introduces a license incompatibility.
The sanlock developers have now rearranged the code such that there
is a 'sanlock_client.so' which is LGPLv2+ while their daemon remains
GPLv2-only. To use the new client library we need to call the new
sanlock_init and sanlock_align APIs instead of sanlock_direct_init
and sanlock_direct_align. These APIs calls are now routed via the
sanlock daemon, instead of doing direct I/O calls to disk.

For all this we require sanlock >= 1.8

* configure.ac: Check for sanlock_client.so instead of sanlock.so
  and fix various comments
* libvirt.spec.in: Mandate sanlock >= 1.8
* src/Makefile.am: Link to -lsanlock_client
* src/locking/lock_driver_sanlock.c: Use sanlock_init and
  sanlock_align

Libvirt loads the domain conf from status XML if it's running when
starting up. The problem is there is no record of the original conf.
(dom->newDef is NULL here).

So libvirt won't be able to restore the domain conf to original one
when destroying/shutdown. E.g.

1) attach a device without "--persistent"
2) restart libvirtd
3) destroy domain
4) start domain

One will see the the disk still exists.

This patch is to fix the peoblem by assigning persistent domain conf
to dom->newDef if it's NULL and the domain is running.

Doing libvirt_parthelper on an not existed device path will get
an unfriendly error message. This patch is to prohibit it.

Silently setting "timeout" as -1 if the specified value is invalid
is a bit confused.

Patch 79cf07af missed one instance of "devname" in source for RPC code
generator.

Translators are likely to botch trailing spacing; by doing the
formatting outside of the translation, we can generally get
better alignment.  Also, for consistency, use 'bytes read' to
match 'bytes written'.

* tools/virsh.c (domblkstat_output): Drop trailing space. Tweak
rd_bytes output.
(cmdDomblkstat, DOMBLKSTAT_LEGACY_PRINT): Update formatting.

Virsh man page lists driver types to be used with attach-device
command, but does not specify that those are usable only with the XEN
Hypervisor.

This patch adds statement, that those options specified are applicable
only on the Xen hypervisor and adds option usable with qemu emulator.

This patch also changes type of error returned by QEMU driver if the
user specifies incompatible driver type from VIR_ERR_INTERNAL_ERROR to
VIR_ERR_CONFIG_UNSUPPORTED.

Alex recently committed some patches with just an email instead
of a preferred name; this fixes things so 'git shortlog' gives
nicer output.

* .mailmap: Update.

Users of virsh complain that output of the domblkstat command
is not intuitive enough. This patch adds explanation of fields
returned by this command to the help section for domblkstat and
the man page of virsh. Also a switch --human is added for
domblkstat that prints the fields with more descriptive
texts.

This patch also changes sequence of the output fields and their
names back to the order and spelling established by previous
versions of virsh to maintain compatibility with scripts.

Example of ordered and "translated" output:

PRE-patch:

virsh # domblkstat 1 vda
vda wr_bytes 5170176
vda wr_operations 511
vda rd_bytes 82815488
vda rd_operations 3726

POST-patch:

virsh # domblkstat 1 vda
vda rd_req 3726
vda rd_bytes 82815488
vda wr_req 478
vda wr_bytes 4965376

Example of human readable output:

virsh # domblkstat 1 vda --human
Device: vda
 number of read operations:      3726
 number of read bytes:           82815488
 number of write operations:     478
 number of bytes written:        4965376

https://bugzilla.redhat.com/show_bug.cgi?id=731656

* src/vmx/vmx.c: fix memory leak, 'def' has a initial value 'NULL', so
  'goto cleanup' is perfected instead of adding a virConfFree before
  'return NULL'.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Leak in pciGetVirtualFunctionIndex present since commit 17d64cab.

* src/util/pci.c: fix memory leak.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Leak present since introduction of remoteDomainBuildEventGraphics
in commit 987e31ed.

* src/remote/remote_driver.c: fix memory leak.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Leak introduced in commit 036ad505, affects only error case.

* src/qemu/qemu_command.c: fix memory leak.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Introduced in commit efa7fc9f.

* daemon/remote.c: fix memory leak in remoteDispatchDomainBlockStatsFlags
Signed-off-by: NAlex Jia <ajia@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=738725 documents that
'yum install libvirt' in Fedora 16 is rather noisy.  This fixes
the problems.

* libvirt.spec.in (%post client): Silence chkconfig warning about
SysV services.
(%post) [with_cgconfig]: Drop for Fedora 15 and newer, where
systemd does this automatically.

For all types of disks other than qcow2, we were requesting that
SELinux labeling visit the new file as if it were qcow2, which
means labeling would try to find the backing files of an empty file.
And for a pre-existing qcow2 disk, we were passing NULL, which meant
that labelling tried to probe the file type (and if probing is
disabled, per the default qemu.conf, this made snapshots fail).
What we really want is to make SELinux labeling visit the new
file as raw; it will later be converted to qcow2 if qemu successfully
made the snapshot.

* src/qemu/qemu_driver.c
(qemuDomainSnapshotCreateSingleDiskActive): Force SELinux labeling
to avoid probe of new file.

For external snapshots to be useful on persistent domains, we must
alter the persistent definition alongside the running definition.
Thanks to the possibility of disk hotplug as well as of edits that
only affect the persistent xml, we can't assume that vm->def and
vm->newDef have the same disk at the same index, so we can only
update the persistent copy if the device destination matches up.

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateDiskActive)
(qemuDomainSnapshotCreateSingleDiskActive): Also affect newDef, if
present.

Mingw lacks fsync, but gnulib provides that.  Meanwhile, gnulib does
not (yet) provide fdatasync, so this is a quick hack to fake that
function on MacOS X; we can revert this configure change once gnulib
gives us a real module.

We have been implicitly relying on gnulib's largefile module being
pulled in by other modules, but it's better to make that explicit.

* bootstrap.conf (gnulib_modules): Add fsync.  Make largefile use
explicit.
* configure.ac (AC_CHECK_FUNCS_ONCE): Check for fdatasync, and
fake it with fsync when not present.

When libvirt calls virInitialize it creates a thread local
for the virErrorPtr storage, and registers a callback to
cleanup memory when a thread exits. When libvirt is dlclose()d
or otherwise made non-resident, the callback function is
removed from memory, but the thread local may still exist
and if a thread later exists, it will invoke the callback
and SEGV. There may also be other thread locals with callbacks
pointing to libvirt code, so it is in general never safe to
unload libvirt.so from memory once initialized.

To allow dlclose() to succeed, but keep libvirt.so resident
in memory, link with '-z nodelete'. This issue was first
found with the libvirt CIM provider, but can potentially
hit many of the dynamic language bindings which all ultimately
involve dlopen() in some way, either on libvirt.so itself,
or on the glue code for the binding which in turns links
to libvirt

* configure.ac, src/Makefile.am: Ensure libvirt.so is linked
  with -z nodelete
* cfg.mk, .gitignore, tests/Makefile.am, tests/shunloadhelper.c,
  tests/shunloadtest.c: A test case to unload libvirt while
  a thread is still running.

* src/qemu/qemu_command.c: missing return value check.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Qemu sends STOP event as part of the shutdown process. Detect such STOP
event and consider shutdown to be reason of emitting such event. That's
the best we can do until qemu provides us the reason directly in STOP
event. This allows us to report shutdown reason for paused state so that
apps can detect domains that failed to finish the shutdown process
(e.g., because qemu is buggy and doesn't exit on SIGTERM or it is
blocked in flushing disk buffers).

Ever since we introduced fake reboot, we call qemuProcessKill as a
reaction to SHUTDOWN event. Unfortunately, qemu doesn't guarantee it
flushed all internal buffers before sending SHUTDOWN, in which case
killing the process forcibly may result in (virtual) disk corruption.

By sending just SIGTERM without SIGKILL we give qemu time to to flush
all buffers and exit. Once qemu exits, we will see an EOF on monitor
connection and tear down the domain. In case qemu ignores SIGTERM or
just hangs there, the process stays running but that's not any different
from a possible hang anytime during the shutdown process so I think it's
just fine.

Also qemu (since 0.14 until it's fixed) has a bug in SIGTERM processing
which causes it not to exit but instead send new SHUTDOWN event and keep
waiting. I think the best we can do is to ignore duplicate SHUTDOWN
events to avoid a SHUTDOWN-SIGTERM loop and leave the domain in paused
state.

When a domain is rebooted using libvirt API, we use fake reboot
consisting of shutting down and resetting the domain. Thus we see a
SHUTDOWN event and set gotShutdown flag. But we never reset it back and
if the domain crashes after it was rebooted this way, we consider it was
a normal shutdown and not a crash.

Commit 4454a9ef changed shutoff reason
from VIR_DOMAIN_SHUTOFF_CRASHED to VIR_DOMAIN_SHUTOFF_FAILED in case we
see an unexpected EOF on monitor connection. But FAILED reason is
dedicated for domains that fail to start. CRASHED reason is the right
one to use in this situation.

This reverts commit 799912fa; now
that the rpc regression is fixed, virsh no longer needs the special
case here.

Libvirt special-cases a specific VIR_ERR_RPC from the remote driver
back into VIR_ERR_NO_SUPPORT on the client, so that clients can
handle missing rpc functions the same whether the hypervisor driver
is local or remote.  However, commit c1b22644 introduced a regression:
VIR_FROM_THIS changed from VIR_FROM_REMOTE to VIR_FROM_RPC, so the
special casing no longer works if the server uses the newer error
domain.

* src/rpc/virnetclientprogram.c
(virNetClientProgramDispatchError): Also cater to 0.9.3 and newer.

This patch fixes the bug shown in bugzilla 738778. It's not an nwfilter problem but a connection sharing / closure issue.

https://bugzilla.redhat.com/show_bug.cgi?id=738778

Depending on the speed / #CPUs of the machine you are using you may not see this bug all the time.

* conf/domain_conf.c: allocate memory to def->redirdevs in
  virDomainDefParseXML such as VIR_ALLOC_N(def->redirdevs, n),
  however, virDomainDefFree(def) hasn't released these memory.

* Detected in valgrind run:

==19820== 209 (16 direct, 193 indirect) bytes in 1 blocks are definitely lost in loss record 25 of 26
==19820==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==19820==    by 0x4A13AF: virAllocN (memory.c:129)
==19820==    by 0x4D4A0E: virDomainDefParseXML (domain_conf.c:7258)
==19820==    by 0x4D4C93: virDomainDefParseNode (domain_conf.c:7512)
==19820==    by 0x4D562F: virDomainDefParse (domain_conf.c:7465)
==19820==    by 0x415863: testCompareXMLToXMLFiles (qemuxml2xmltest.c:35)
==19820==    by 0x415982: testCompareXMLToXMLHelper (qemuxml2xmltest.c:80)
==19820==    by 0x416D31: virtTestRun (testutils.c:140)
==19820==    by 0x415604: mymain (qemuxml2xmltest.c:192)
==19820==    by 0x416437: virtTestMain (testutils.c:689)
==19820==    by 0x3CA7A1ECDC: (below main) (in /lib64/libc-2.12.so)
==19820==
==19820== LEAK SUMMARY:
==19820==    definitely lost: 16 bytes in 1 blocks
==19820==    indirectly lost: 193 bytes in 5 blocks
==19820==      possibly lost: 0 bytes in 0 blocks
==19820==    still reachable: 1,054 bytes in 21 blocks

* How to reproduce?
% valgrind -v --leak-check=full ./tests/qemuxml2xmltest
Signed-off-by: NAlex Jia <ajia@redhat.com>

Mac OS X 10.6. Snow Leopard and probably other do not provide a mkfs
command to create filesystems. Macro MKFS then remained undefined and
did not provide any substitute, so that build failed on a missing
argument.

Struct virStoragePoolProbeResult was compiled in conditionaly, but
virStorageBackendFileSystemProbe used it unconditionaly. This patch
exempts the struct from conditional include.

/usr/lib/stdlib.h in Mac OS X and probably also in BSD's
exports this symbol :(