'make check' fails since commit 470d5c46 on any system with dash
as /bin/sh, because '<<<' is a bash extension.  For example:

nwfilterschematest: 23: /home/eblake/libvirt/tests/schematestutils.sh: Syntax error: redirection unexpected

Also, there is no need to spawn a grep process when shell globbing
can do the same.

* tests/schematestutils.sh: Replace bashism and subprocess with a
faster and portable construct.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 1d21b884)

Commit 632f78ca introduced a regression which causes schedinfo being
unable to set some parameters.  When migrating to priv->cgroup there
was missing variable left out and due to passed NULL to underlying
function, the setting failed.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=963592

CVE-2013-1962

remoteDispatchStoragePoolListAllVolumes wasn't freeing the pool.
The pool also held a reference to the connection, preventing it from
getting freed and closing the netcf interface driver, which held two
sockets open.
(cherry picked from commit ca697e90)

bz #960879

In b2878ed8 we added the O_NOCTTY
flag when opening files in the stream code. Unfortunately a later
piece of code was comparing the flags == O_RDONLY, without masking
out the non-access mode flags. This broke the iohelper when used
with streams for read, since it caused us to attach the stream
output pipe to the stream input FD instead of output FD :-(
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit a2214c52)

This resolves:

     https://bugzilla.redhat.com/show_bug.cgi?id=851411
     https://bugzilla.redhat.com/show_bug.cgi?id=955500

The first problem was that virFileOpenAs was returning fd (-1) in one
of the error cases rather than ret (-errno), so the caller thought
that the error was EPERM rather than ENOENT.

The second problem was that some log messages in the general purpose
qemuOpenFile() function would always say "Failed to create" even if
the caller hadn't included O_CREAT (i.e. they were trying to open an
existing file).

This fixes virFileOpenAs to jump down to the error return (which
returns ret instead of fd) in the previously mentioned incorrect
failure case of virFileOpenAs(), removes all error logging from
virFileOpenAs() (since the callers report it), and modifies
qemuOpenFile to appropriately use "open" or "create" in its log
messages.

NB: I seriously considered removing logging from all callers of
virFileOpenAs(), but there is at least one case where the caller
doesn't want virFileOpenAs() to log any errors, because it's just
going to try again (qemuOpenFile()). We can't simply make a silent
variation of virFileOpenAs() though, because qemuOpenFile() can't make
the decision about whether or not it wants to retry until after
virFileOpenAs() has already returned an error code.

Likewise, I also considered changing virFileOpenAs() to return -1 with
errno set on return, and may still do that, but only as a separate
patch, as it obscures the intent of this patch too much.
(cherry picked from commit a2c1bedb)

(cherry picked from commit 413274f6)

The controller element supports non-disk controller types too.

https://bugzilla.redhat.com/show_bug.cgi?id=960958
(cherry picked from commit c075f89f)

Print an error instead of crashing when a TPM device without
a backend is specified.

Add a test for tpm device with no backend, which should fail
with a parse error.

https://bugzilla.redhat.com/show_bug.cgi?id=961252
(cherry picked from commit 1d96440a)

Currently, using an invalid XML in tests fails, because
the schema test expects all of them to be valid.

Treat files with -invalid.xml suffix as invalid and expect
them to fail validation.
(cherry picked from commit 470d5c46)

VFIO device assignment requires a cgroup ACL to be setup for access to
the /dev/vfio/nn "group" device for any devices that will be assigned
to a guest. In the case of a host device that is allocated from a
pool, it was being allocated during qemuBuildCommandLine(), which is
called by qemuProcessStart() *after* the all-encompassing
qemuSetupCgroup() was called, meaning that the standard Cgroup ACL
setup wasn't creating ACLs for these devices allocated from pools.

One possible solution was to manually add a single ACL down inside
qemuBuildCommandLine() when networkAllocateActualDevice() is called,
but that has two problems: 1) the function that adds the cgroup ACL
requires a virDomainObjPtr, which isn't available in
qemuBuildCommandLine(), and 2) we really shouldn't be doing network
device setup inside qemuBuildCommandLine() anyway.

Instead, I've created a new function called
qemuNetworkPrepareDevices() which is called just before
qemuPrepareHostDevices() during qemuProcessStart() (explanation of
ordering in the comments), i.e. well before the call to
qemuSetupCgroup(). To minimize code churn in a patch that will be
backported to 1.0.5-maint, qemuNetworkPrepareDevices only does
networkAllocateActualDevice() and the bare amount of setup required
for type='hostdev network devices, but it eventually should do *all*
device setup for guest network devices.

Note that some of the code that was previously needed in
qemuBuildCommandLine() is no longer required when
networkAllocateActualDevice() is called earlier:

 * qemuAssignDeviceHostdevAlias() is already done further down in
   qemuProcessStart().

 * qemuPrepareHostdevPCIDevices() is called by
   qemuPrepareHostDevices() which is called after
   qemuNetworkPrepareDevices() in qemuProcessStart().

As hinted above, this new function should be moved into a separate
qemu_network.c (or similarly named) file along with
qemuPhysIfaceConnect(), qemuNetworkIfaceConnect(), and
qemuOpenVhostNet(), and expanded to call those functions as well, then
the nnets loop in qemuBuildCommandLine() should be reduced to only
build the commandline string (which itself can be in a separate
qemuInterfaceBuilldCommandLine() function as suggested by
Michal). However, this will require storing away an array of tapfd and
vhostfd that are needed for the commandline, so I would rather do that
in a separate patch and leave this patch at the minimum to fix the
bug.
(cherry picked from commit 8cd40e7e)

In the non linux case some callers like gather_scsi_host_caps needed the
return code of -1 while others like update_caps needed an empty
statement (to avoid a "statement without effect" warning). This is much
simpler solved by using a function instead of a define.
(cherry picked from commit 58662f44)

since it uses glibc's rpcgen.
(cherry picked from commit b562d7b7)

The lockd plugin for the lock manager was not correctly
handling the release of resource locks. This meant that
during migration, or when pausing a VM, the locks would
not get released. This in turn made it impossible to
resume the domain, or finish migration
(cherry picked from commit 8dc93ffa)

POSIX says pthread_t is opaque.  We can't guarantee if it is scaler
or a pointer, nor what size it is; and BSD differs from Linux.
We've also had reports of gcc complaining on attempts to cast it,
if we use a cast to the wrong type (for example, pointers have to be
cast to void* or intptr_t before being narrowed; while casting a
function return of scalar pthread_t to void* triggers a different
warning).

Give up on casts, and use unions to get at decent bits instead.  And
rather than futz around with figuring which 32 bits of a potentially
64-bit pointer are most likely to be unique, convert the rest of
the code base to use 64-bit values when using a debug id.

Based on a report by Guido Günther against kFreeBSD, but with a
fix that doesn't regress commit 4d970fd2 for FreeBSD.

* src/util/virthreadpthread.c (virThreadSelfID, virThreadID): Use
union to get at a decent bit representation of thread_t bits.
* src/util/virthread.h (virThreadSelfID, virThreadID): Alter
signature.
* src/util/virthreadwin32.c (virThreadSelfID, virThreadID):
Likewise.
* src/qemu/qemu_domain.h (qemuDomainJobObj): Alter type of owner.
* src/qemu/qemu_domain.c (qemuDomainObjTransferJob)
(qemuDomainObjSetJobPhase, qemuDomainObjReleaseAsyncJob)
(qemuDomainObjBeginNestedJob, qemuDomainObjBeginJobInternal): Fix
clients.
* src/util/virlog.c (virLogFormatString): Likewise.
* src/util/vireventpoll.c (virEventPollInterruptLocked):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 22d12905)

If an early dispatch check caused a jump to the 'cleanup' branch
then virTypeParamsFree() would be called with an uninitialized
'nparams' variable. Fortunately 'params' is initialized to NULL,
so the uninitialized 'nparams' variable would not be used.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 31dbbb66)

Commit 776d49f4 added a static function that is only called
conditionally; leading to this compile error on mingw:

  CC       libvirt_util_la-virprocess.lo
../../src/util/virprocess.c:624:26: error: 'struct rlimit' declared inside parameter list [-Werror]
../../src/util/virprocess.c:624:26: error: its scope is only this definition or declaration, which is probably not what you want [-Werror]
../../src/util/virprocess.c:622:1: error: 'virProcessPrLimit' defined but not used [-Werror=unused-function]

* src/util/virprocess.c (virProcessPrLimit): Only declare
virProcessPrLimit when used.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 05f79a38)

The F_DUPFD_CLOEXEC operation with fcntl() expects a single
int argument, specifying the minimum FD number for the newly
dup'd file descriptor. We were not specifying that causing
random stack data to be accessed as the FD number. Sometimes
that worked, sometimes it didn't.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit d6670a64)

https://bugzilla.redhat.com/show_bug.cgi?id=924501 tracks a
problem that occurs if uid 107 is already in use at the time
libvirt is first installed.  In response that problem, Fedora
packaging guidelines were recently updated.  This fixes the
spec file to comply with the new guidelines:
https://fedoraproject.org/wiki/Packaging:UsersAndGroups

* libvirt.spec.in (daemon): Follow updated Fedora guidelines.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit a2584d58)

'make distcheck' complained:

ERROR: files left in build directory after distclean:
./python/libvirt.pyc
./tests/commandhelper.log

Problems introduced in commits f0154959 and 25ea8e47 (both v1.0.3).

* tests/commandtest.c (test21): Check (and clean) log file.
* tests/commanddata/test21.log: New file.
* python/Makefile.am (CLEANFILES): Clean up compiled python files.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit c21c38d7)

On a mingw build, 'make distcheck' fails with:

  GEN      libvirt_qemu.def
make[3]: *** No rule to make target `../../src/libvirt_lxc.syms', needed by `libvirt_lxc.def'.  Stop.

I traced it to a missing entry in EXTRA_DIST.  But rather than keep
the entire list in sync, it is easier to list the three syms files
that drive .so files directly, and then reuse existing makefile
variables for the remaining files (that is, I validated that all
remaining files are added to SYM_FILES, possibly via USED_SYM_FILES,
according to makefile conditionals).

Problem introduced in commit 3d1596b0 (v1.0.2).

* src/Makefile.am (EXTRA_DIST): Ensure all syms files are shipped.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit ed11ab93)

I must have looked at this a couple dozen times before I noticed it
had "!=" instead of "==". Not doing this setup prevented qemu from
doing anything with the vfio group device.
(cherry picked from commit 52ba0f6e)

The libvirt 1.0.5 tarball is missing a file which renders 'make
check' broken; first reported on list by Guido Günther.

* python/Makefile.am (EXTRA_DIST): Hoist sanitytest.py out of
HAVE_PYTHON conditional.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 5a3487da)

introduced by dcf97846

To trigger this cppi needs to be installed.
(cherry picked from commit ca75c443)

introduced by dcf97846
(cherry picked from commit a1365d73)

This should resolve https://bugzilla.redhat.com/show_bug.cgi?id=958907

Recent new addition of code to read/write active network state to the
NETWORK_STATE_DIR in the network driver broke startup for
qemu:///session. The network driver had several state file paths
hardcoded to /var, which could never possibly work in session mode.

This patch modifies *all* state files to use a variable string that is
set differently according to whether or not we're running
privileged. (It turns out that logDir was never used, so it's been
completely eliminated.)

There are very definitely other problems preventing dnsmasq and radvd
from running in non-privileged mode, but it's more consistent to have
the directories used by them be determined in the same fashion.

NB: I've noted before that the network driver is storing its state
(including dnsmasq and radvd state) in /var/lib, while qemu stores its
state in /var/run. It would probably have been better if the two
matched, but it's been this way for a long time, and changing it would
break running installations during an upgrade, so it's best to just
leave it as it is.
(cherry picked from commit 2ffd87d8)

Conflicts:
	src/network/bridge_driver.c - code removed by this patch had
                                      been modified upstream.

* configure.ac docs/news.html.in libvirt.spec.in: updated for the release
* po/*.po*: pulled and merged a number of new localization updates

When MAXHOSTNAMELEN is set we have to take it's value into account.
Otherwise the build fails on kFreeBSD (FreeBSD kernel and GNU userland)

virPCIDeviceReattach and virPCIDeviceUnbindFromStub (called by
virPCIDeviceReattach) had previously required the name of the stub
driver as input. This is unnecessary, because the name of the driver
the device is currently bound to can be found by looking at the link:

  /sys/bus/pci/dddd:bb:ss.ff/driver

Instead of requiring that the name of the expected stub driver name
and only unbinding if that one name is matched, we no longer take a
driver name in the arglist for either of these
functions. virPCIDeviceUnbindFromStub just compares the name of the
currently bound driver to a list of "well known" stubs (right now
contains "pci-stub" and "vfio-pci" for qemu, and "pciback" for xen),
and only performs the unbind if it's one of those devices.

This allows virsh nodedevice-reattach to work properly across a
libvirtd restart, and fixes a couple of cases where we were
erroneously still hard-coding "pci-stub" as the drive name.

For some unknown reason, virPCIDeviceReattach had been calling
modprobe on the stub driver prior to unbinding the device. This was
problematic because we no longer know the name of the stub driver in
that function. However, it is pointless to probe for the stub driver
at that time anyway - because the device is bound to the stub driver,
we are guaranteed that it is already loaded, and so that call to
modprobe has been removed.

Conditional BuildRequires: should be at the top level, rather
than appearing in conditional sub-package sections.  This
appears to be the only offender.

* libvirt.spec.in (BuildRequires): Move libblkid-devel into
correct area.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit cc6d19f3 added text containing "<code>snapshot<code>" to
formatsnapshot.html.in. The closing tag is missing '/' which causes
the documentation to misrender.

Python code generator "generate_source" section that handles
code generation to "free" inherited objects needs to generate
DISPATCH_FREE calls for all extended_by objects.

For s390 we don't want to have a default USB device generated even
if QEMU is silently tolerating -usb on the command line. This may change
in the future.
Another reason to avoid the USB controller is that it implies a PCI
bus which might cause a regression at some later point in time.
The following change will set the USB controller model to 'none'
unless a model or address has been specified, which can be the case
if a legacy definition is loaded or the XML writer knows what
she/he's doing.
Requiring the user to explicitly disable USB on systems not supporting
it seems cumbersome.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Add a line saying that no USB controllers are generated by default
on s390.

Commit eca3fdf7 inadvertantly caused a failure to start for any domain
with the following in its config:

    <graphics type='spice' autoport='yes'/>

The problem is that when tlsPort == 0 and defaultMode == "any" (which
is the default for defaultMode), this would be flagged in the code as
"needTLSPort", and if there was then no spice tls config, the new
error+fail would happen.

This patch checks for the case of defaultMode == "any", and in that
case simply doesn't allocate a TLS port (since that's probably not
what the user wanted, and it would have failed later anyway.). It does
leave the error in place for cases when the user specifically asked to
use tls in one way or another, though.

On cygwin, compilation failed because SIOCSIFHWADDR is undefined.

* src/util/virnetdev.c (virNetDevSetMAC): Cygwin can query but not
set mac address.
Signed-off-by: NEric Blake <eblake@redhat.com>

As a result of commit id '19c345f2', 'make -C tests valgrind' has the
following for qemuxml2argvtest:

==22482== 197 (80 direct, 117 indirect) bytes in 1 blocks are definitely lost in loss record 101 of 120
==22482==    at 0x4A06B6F: calloc (vg_replace_malloc.c:593)
==22482==    by 0x4C6F301: virAlloc (viralloc.c:124)
==22482==    by 0x4C840FC: virSaveLastError (virerror.c:308)
==22482==    by 0x431882: qemuBuildCommandLine (qemu_command.c:8204)
==22482==    by 0x41E8F0: testCompareXMLToArgvHelper (qemuxml2argvtest.c:155)
==22482==    by 0x41FE9F: virtTestRun (testutils.c:157)
==22482==    by 0x419DEB: mymain (qemuxml2argvtest.c:654)
==22482==    by 0x4204DA: virtTestMain (testutils.c:719)
==22482==    by 0x39D0821A04: (below main) (libc-start.c:225)
==22482==

Return 3 when the service is stopped, whether there
are saved guests or not, to conform with the LSB standards:

http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/iniscrptact.html

FreeBSD (and maybe other BSDs) have different member
names in struct ifreq when compared to Linux, such as:

 - uses ifr_data instead of ifr_newname for setting
   interface names
 - uses ifr_index instead of ifr_ifindex for interface
   index

Also, add a check for SIOCGIFHWADDR for virNetDevValidateConfig().

Use AF_LOCAL if AF_PACKET is not available.
Signed-off-by: NEric Blake <eblake@redhat.com>

If virPCIDeviceGetVFIOGroupDev() failed,
virSecurity*(Set|Restore)HostdevLabel() would fail to free a
virPCIDevice that had been allocated.

These leaks were all introduced (by me) very recently, in commit
f0bd70a9.