This patch adds a netlink callback when migrating a VEPA enabled
virtual machine.  It fixes a Bug where a VM would not request a port
association when it was cleared by lldpad.

This patch requires the latest git version of lldpad to work.
Signed-off-by: ND. Herrendoerfer <d.herrendoerfer@herrendoerfer.name>
(cherry picked from commit 997366ca)

If dynamic_ownership is off and we are creating a file on NFS
we force chown. This will fail as chown/chmod are not supported
on NFS. However, with no dynamic_ownership we are not required
to do any chown.
(cherry picked from commit b1256816)

* daemon/libvirtd-config.c (daemonConfigFree): fix memory leaks.

How to reproduce?

% make && make -C tests check TESTS=libvirtdconftest
% cd tests && valgrind -v --leak-check=full ./libvirtdconftest

actual result:

==11008== 185 bytes in 5 blocks are definitely lost in loss record 3 of 5
==11008==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==11008==    by 0x39CF07F6E1: strdup (strdup.c:43)
==11008==    by 0x406626: daemonConfigLoadOptions (libvirtd-config.c:438)
==11008==    by 0x406800: daemonConfigLoadData (libvirtd-config.c:492)
==11008==    by 0x403CCF: testCorrupt (libvirtdconftest.c:110)
==11008==    by 0x404FAD: virtTestRun (testutils.c:145)
==11008==    by 0x403A34: mymain (libvirtdconftest.c:219)
==11008==    by 0x404687: virtTestMain (testutils.c:700)
==11008==    by 0x39CF01ECDC: (below main) (libc-start.c:226)
==11008==
==11008== LEAK SUMMARY:
==11008==    definitely lost: 185 bytes in 5 blocks
Signed-off-by: NAlex Jia <ajia@redhat.com>
(cherry picked from commit d0eaf4b1)

Conflicts:

	daemon/libvirtd-config.c

The new safe console handling introduced a possibility to deadlock the
qemu driver when a new console connection forcibly disconnects a
previous console stream that belongs to an already closed connection.

The virStreamFree function calls subsequently a the virReleaseConnect
function that tries to lock the driver while discarding the connection,
but the driver was already locked in qemuDomainOpenConsole.

Backtrace of the deadlocked thread:
0  0x00007f66e5aa7f14 in __lll_lock_wait () from /lib64/libpthread.so.0
1  0x00007f66e5aa3411 in _L_lock_500 () from /lib64/libpthread.so.0
2  0x00007f66e5aa322a in pthread_mutex_lock () from/lib64/libpthread.so.0
3  0x0000000000462bbd in qemudClose ()
4  0x00007f66e6e178eb in virReleaseConnect () from/usr/lib64/libvirt.so.0
5  0x00007f66e6e19c8c in virUnrefStream () from /usr/lib64/libvirt.so.0
6  0x00007f66e6e3d1de in virStreamFree () from /usr/lib64/libvirt.so.0
7  0x00007f66e6e09a5d in virConsoleHashEntryFree () from/usr/lib64/libvirt.so.0
8  0x00007f66e6db7282 in virHashRemoveEntry () from/usr/lib64/libvirt.so.0
9  0x00007f66e6e09c4e in virConsoleOpen () from /usr/lib64/libvirt.so.0
10 0x00000000004526e9 in qemuDomainOpenConsole ()
11 0x00007f66e6e421f1 in virDomainOpenConsole () from/usr/lib64/libvirt.so.0
12 0x00000000004361e4 in remoteDispatchDomainOpenConsoleHelper ()
13 0x00007f66e6e80375 in virNetServerProgramDispatch () from/usr/lib64/libvirt.so.0
14 0x00007f66e6e7ae11 in virNetServerHandleJob () from/usr/lib64/libvirt.so.0
15 0x00007f66e6da897d in virThreadPoolWorker () from/usr/lib64/libvirt.so.0
16 0x00007f66e6da7ff6 in virThreadHelper () from/usr/lib64/libvirt.so.0
17 0x00007f66e5aa0c5c in start_thread () from /lib64/libpthread.so.0
18 0x00007f66e57e7fcd in clone () from /lib64/libc.so.6

* src/qemu/qemu_driver.c: qemuDomainOpenConsole()
        -- unlock the qemu driver right after acquiring the domain
        object
(cherry picked from commit 3d3de46a)

I noticed these compiler warnings when building for the s390 architecture.

* src/node_device/node_device_udev.c (udevDeviceMonitorStartup):
Mark unused variable.
* src/nodeinfo.c (linuxNodeInfoCPUPopulate): Avoid unused variable.
(cherry picked from commit 9011a494)

The documentation for the flag doesn't clearly state that the flag only
enhances the output and the user needs to specify other flags to list
inactive domains, that are enhanced by this flag.
(cherry picked from commit cc510065)

(cherry picked from commit 20171c8d)

Below code failed to compile on a 32 bit machine with error

typewrappers.c: In function 'libvirt_intUnwrap':
typewrappers.c:135:5: error: logical 'and' of mutually exclusive tests is always false [-Werror=logical-op]
cc1: all warnings being treated as errors

The patch fixes this error.

(cherry picked from commit 4e9bb1df)

I found typo in UML driver.

MATSUDA Daiki
(cherry picked from commit 257191c9)

Leak introduced in commit 0436d328.  If we allocate an actions array,
but fail early enough to never consume it with the qemu monitor
transaction call, we leaked memory.

But our semantics of making the transaction command free the caller's
memory is awkward; avoiding the memory leak requires making every
intermediate function in the call chain check for error.  It is much
easier to fix things so that the function that allocates also frees,
while the call chain leaves the caller's data intact.  To do that,
I had to hack our JSON data structure to make it easy to protect a
portion of an arbitrary JSON tree from being freed.

* src/util/json.h (virJSONType): Name the enum.
(_virJSONValue): New field.
* src/util/json.c (virJSONValueFree): Use it to protect a portion
of an array.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONTransaction): Avoid
freeing caller's data.
* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateDiskActive):
Free actions array on failure.
(cherry picked from commit 14135609)

We can tell qemuDomainSnapshotFSThaw if we want it to report errors or
not. However, if we don't want to and an error has been already set by
previous qemuReportError() we must keep copy of that error not just a
pointer to it. Otherwise, it get overwritten if FSThaw reports an error.
(cherry picked from commit 650da0e9)

This causes an implicit vkbd device to be added which takes
6min to finally fail being initialized in the guest.

http://lists.xen.org/archives/html/xen-devel/2012-04/msg00409.htmlSigned-off-by: NStefan Bader <stefan.bader@canonical.com>
(cherry picked from commit fb98da00)

gcc 4.7 warns about uninitialized struct members

* tests/testutilsqemu.c (testQemuCapsInit): Populate new members.
* tests/viruritest.c (mymain): Likewise.
(cherry picked from commit 72505073)

When building on Fedora 17 (which uses gcc 4.7.0) with -O0 in CFLAGS,
three of the tests failed to compile.

cputest.c and qemuxml2argvtest.c had non-static structs defined
inside the macro that was being repeatedly invoked. Due to some so-far
unidentified change in gcc, the stack space used by variables defined
inside { } is not recovered/re-used when the block ends, so all these
structs have become additive (this is the same problem worked around
in commit cf57d345). Fortunately, these two files could be fixed with
a single line addition of "static" to the struct definition in the
macro.

virnettlscontexttest.c was a bit different, though. The problem structs
in the do/while loop of macros had non-constant initializers, so it
took a bit more work and piecemeal initialization instead of member
initialization to get things to be happy.

In an ideal world, none of these changes should be necessary, but not
knowing how long it will be until the gcc regressions are fixed, and
since the code is just as correct after this patch as before, it makes
sense to fix libvirt's build for -O0 while also reporting the gcc
problem.
(cherry picked from commit 06a1a45c)

This got dropped with 300e60e1

Cheers,
 -- Guido
(cherry picked from commit 4eb1c256)

This bug resolves https://bugzilla.redhat.com/show_bug.cgi?id=810100

rpm builds for i686 were failing with a segfault in
networkxml2argvtest. Running under valgrind showed that a region of
memory was being referenced after it had been freed (as the result of
realloc - see the valgrind report in the BZ).

The problem (in replaceTokens() - added in commit 22ec60, meaning this
bug was in 0.9.10 and 0.9.11) was that the pointers token_start and
token_end were being computed based on the value of *buf, then *buf
was being realloc'ed (potentially moving it), then token_start and
token_end were used without recomputing them to account for movement
of *buf.

The solution is to change the code so that token_start and token_end
are offsets into *buf rather than pointers. This way there is only a
single pointer to the buffer, and nothing needs readjusting after a
realloc. (You may note that some uses of token_start/token_end didn't
need to be changed to add in "*buf +" - that's because there ended up
being a +*buf and -*buf which canceled each other out).

DV gets the credit for finding this bug and pointing out the valgrind
report.
(cherry picked from commit bde32b1a)

Detected by valgrind. Leaks are introduced in commit b22eaa75.

* src/conf/domain_conf.c (virDomainDiskDefParseXML): fix memory leaks.

How to reproduce?

% make && make -C tests check TESTS=qemuxml2argvtest
% cd tests && valgrind -v --leak-check=full ./qemuxml2argvtest

actual result:

==2143== 12 bytes in 2 blocks are definitely lost in loss record 74 of 179
==2143==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==2143==    by 0x39D90A67DD: xmlStrndup (xmlstring.c:45)
==2143==    by 0x4F5EC0: virDomainDiskDefParseXML (domain_conf.c:3438)
==2143==    by 0x502F00: virDomainDefParseXML (domain_conf.c:8304)
==2143==    by 0x505FE3: virDomainDefParseNode (domain_conf.c:9080)
==2143==    by 0x5069AE: virDomainDefParse (domain_conf.c:9030)
==2143==    by 0x41CBF4: testCompareXMLToArgvHelper (qemuxml2argvtest.c:105)
==2143==    by 0x41E5DD: virtTestRun (testutils.c:145)
==2143==    by 0x416FA3: mymain (qemuxml2argvtest.c:399)
==2143==    by 0x41DCB7: virtTestMain (testutils.c:700)
==2143==    by 0x39CF01ECDC: (below main) (libc-start.c:226)
Signed-off-by: NAlex Jia <ajia@redhat.com>
(cherry picked from commit 80d476a9)

https://bugzilla.redhat.com/show_bug.cgi?id=809895

Basically, openvz dropped strict version numbering (3.1 vs 3.1.0),
which caused parsing to fail.
(cherry picked from commit 37075dfe)

* configure.ac: Set WITH_SYSCTL only on Linux hosts
* daemon/Makefile.am: Conditionalize install-sysctl using WITH_SYSCTL
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Cc: Jason Helfman <jhelfman@e-e.com>
(cherry picked from commit e7eca6e4)

Every now & then, with parallel builds, we get a failure to
validate hvsupport.html.in.  I eventually noticed that this
is because we get 2 instances of the generator running at
once.

We already list hvsupport.html.in in BUILT_SOURCES but this
was not working. It turns out the flaw is that we were
adding deps to the 'all:' target instead of the 'all-am:'
target. BUILT_SOURCES is a dep of 'all', so any custom
targets written in Makefile.am must use 'all-am:' so that
they don't get run until BUILT_SOURCES are completely
generated

* docs/Makefile.am: s/all/all-am/
(cherry picked from commit 4f4b496e)

(cherry picked from commit f94d9c57)

This symbol is used in the test suites
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 06180ca4)

Some of the test suites use fprintf with format specifiers
that are not supported on Win32 and are not fixed by gnulib.

The mingw32 compiler also has trouble detecting ssize_t
correctly, complaining that 'ssize_t' does not match
'signed size_t' (which it expects for %zd). Force the
cast to size_t to avoid this problem

* tests/testutils.c, tests/testutils.h: Fix printf
  annotation on virTestResult. Use virVasprintf
  instead of vfprintf
* tests/virhashtest.c: Use VIR_WARN instead of fprintf(stderr).
  Cast to size_t to avoid mingw32 compiler bug
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit f48de0f1)

If the daemon is restarted it will lose list of active
USB devices assigned to active domains. Therefore we need
to rebuild this list on qemuProcessReconnect().
(cherry picked from commit ea3bc548)

To prevent assigning one USB device to two domains,
we keep a list of assigned USB devices. On domain
startup - qemuProcessStart() - we insert devices
used by domain into the list but remove them only
on detach-device. Devices are, however, released
on qemuProcessStop() as well.
(cherry picked from commit e2f5dd61)

and add debug message when adding USB device
to the list of active devices.
(cherry picked from commit b2c7b9ee)

* docs/news.html.in: Fix accidental deletion.
(cherry picked from commit ec8552f3)

Void elements should be written with slash *after* the tag name,
not before, so they are not confused with ending tags.
(cherry picked from commit ae277e4e)

Currently, we put no strains on escape sequence possibly leaving users
with console that cannot be terminated. However, not all ASCII
characters can be used as escape sequence. Only those falling in
@ - _ can be; implement and document this constraint.
(cherry picked from commit 7960ce84)

* configure.ac docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: updated a number of languages translation including new
  indian languages and regenerated

This reverts commit 06a0d57f.

Originally, qemuDomainCheckEjectableMedia was entering monitor with qemu
driver lock. Commit 2067e31b, which I
made to fix that, revealed another issue we had (but didn't notice it
since the driver was locked): we didn't set nested job when
qemuDomainCheckEjectableMedia is called during migration. Thus the
original fix I made was wrong.

XenD-3.1 introduced managed domains. HV-domains have rtc_timeoffset
(hgd24f37b31030 from 2007-04-03), which tracks the offset between the
hypervisors clock and the domains RTC, and is persisted by XenD.
In combination with localtime=1 this had a bug until XenD-3.4
(hg5d701be7c37b from 2009-04-01) (I'm not 100% sure how that bug
manifests, but at least for me in TZ=Europe/Berlin I see the previous
offset relative to utc being applied to localtime again, which manifests
in an extra hour being added)

XenD implements the following variants for clock/@offset:
- PV domains don't have a RTC → 'localtime' | 'utc'
- <3.1: no managed domains → 'localtime' | 'utc'
- ≥3.1: the offset is tracked for HV → 'variable'
        due to the localtime=1 bug → 'localtime' | 'utc'
- ≥3.4: the offset is tracked for HV → 'variable'

Current libvirtd still thinks XenD only implements <clock offset='utc'/>
and <clock offset='localtime'/>, which is wrong, since the semantic of
'utc' and 'localtime' specifies, that the offset will be reset on
domain-restart, while with 'variable' the offset is kept. (keeping the
offset over "virsh edit" is important, since otherwise the clock might
jump, which confuses certain guest OSs)

xendConfigVersion was last incremented to 4 by the xen-folks for
xen-3.1.0. I know of no way to reliably detect the version of XenD
(user space tools), which may be different from the version of the
hypervisor (kernel) version! Because of this only the change from
'utc'/'localtime' to 'variable' in XenD-3.1 is handled, not the buggy
behaviour of XenD-3.1 until XenD-3.4.

For backward compatibility with previous versions of libvirt Xen-HV
still accepts 'utc' and 'localtime', but they are returned as 'variable'
on the next read-back from Xend to libvirt, since this is what XenD
implements: The RTC is NOT reset back to the specified time on next
restart, but the previous offset is kept.
This behaviour can be turned off by adding the additional attribute
adjustment='reset', in which case libvirt will report an error instead
of doing the conversion. The attribute can also be used as a shortcut to
offset='variable' with basis='...'.

With these changes, it is also necessary to adjust the xen tests:

"localtime = 0" is always inserted, because otherwise on updates the
value is not changed within XenD.

adjustment='reset' is inserted for all cases, since they're all <
XEND_CONFIG_VERSION_3_1_0, only 3.1 introduced persistent
rtc_timeoffset.

Some statements change their order because code was moved around.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Since Xen 3.1 the clock=variable semantic is supported. In addition to
qemu/kvm Xen also knows about a variant where the offset is relative to
'localtime' instead of 'utc'.

Extends the libvirt structure with a flag 'basis' to specify, if the
offset is relative to 'localtime' or 'utc'.

Extends the libvirt structure with a flag 'reset' to force the reset
behaviour of 'localtime' and 'utc'; this is needed for backward
compatibility with previous versions of libvirt, since they report
incorrect XML.

Adapt the only user 'qemu' to the new name.
Extend the RelaxNG schema accordingly.
Document the new 'basis' attribute in the HTML documentation.
Adapt test for the new attribute.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

https://bugzilla.redhat.com/show_bug.cgi?id=808979

The leak is really in virProcessInfoGetAffinity, as shown in the
valgrind output given in the above bug report - it calls CPU_ALLOC(),
but then fails to call CPU_FREE().

This leak has existed in every version of libvirt since 0.7.5.

Commit 1b1402b9 introduced a regression.  Since older libvirt versions
would silently round memory up (until the previous patch), but populated
current memory based on querying the guest, it was possible to have
dumpxml show cur > max by the amount of the rounding.  For example, if
a user requested 1048570 KiB memory (just shy of 1GiB), the qemu
driver would actually run with 1048576 KiB, and libvirt 0.9.10 would
output a current that was 6KiB larger than the maximum.  Situations
where this could have an impact include, but are not limited to,
migration from old to new libvirt, managedsave in old libvirt and
start in new libvirt, snapshot creation in old libvirt and revert in
new libvirt - without this patch, the new libvirt would reject the
VM because of the rounding discrepancy.

Fix things by adding a fuzz factor, and silently clamp current down to
maximum in that case, rather than failing to reparse XML for an existing
VM.  From a practical standpoint, this has no user impact: 'virsh
dumpxml' will continue to query the running guest rather than rely on
the incoming xml, which will see the currect current value, and even if
clamping down occurs during parsing, it will be by at most the fuzz
factor of a megabyte alignment, and rounded back up when passed back to
the hypervisor.

Meanwhile, we continue to reject cur > max if the difference is beyond
the fuzz factor of nearest megabyte.  But this is not a real change in
behavior, since with 0.9.10, even though the parser allowed it, later
in the processing stream we would reject it at the qemu layer; so
rejecting it in the parser just moves error detection to a nicer place.

* src/conf/domain_conf.c (virDomainDefParseXML): Don't reject
existing XML.
Based on a report by Zhou Peng.

If we round up a user's memory request, we should update the XML
to reflect the actual value in use by the VM, rather than giving
an artificially small value back to the user.

* src/qemu/qemu_command.c (qemuBuildNumaArgStr)
(qemuBuildCommandLine): Reflect rounding back to XML.

Laszlo Ersek pointed out that in trying to convert a long to an
unsigned int, we used:

long long_val = ...;
if ((unsigned int)long_val == long_val)

According to C99 integer promotion rules, the if statement is
equivalent to:

(unsigned long)(unsigned int)long_val == (unsigned long)long_val

since you get an unsigned comparison if at least one side is
unsigned, using the largest rank of the two sides; but on 32-bit
platforms, where unsigned long and unsigned int are the same size,
this comparison is always true and ends up converting negative
long_val into posigive unsigned int values, rather than rejecting
the negative value as we had originally intended (python longs
are unbounded size, and we don't want to do silent modulo
arithmetic when converting to C code).

Fix this by using direct comparisons, rather than casting.

* python/typewrappers.c (libvirt_intUnwrap, libvirt_uintUnwrap)
(libvirt_ulongUnwrap, libvirt_ulonglongUnwrap): Fix conversion
checks.

* libvirt.spec.in: Remove obsolete --with-remote-pid-file arg.
  Add missing %{without_libxl} statement. Fix handling of docs
  in client only build. Put systemtap files in -client RPM
  instead of -daemon RPM
* examples/xml/nwfilter/Makefile.am: Don't install examples if
  nwfilter is disabled.