VIR_APPEND_ELEMENT(array, size, elem) was not safe if the expression
for 'size' had side effects.  While no one in the current code base
was trying to pass side effects, we might as well be robust and
explicitly document our intentions.

* src/util/viralloc.c (virInsertElementsN): Add special case.
* src/util/viralloc.h (VIR_APPEND_ELEMENT): Use it.
(VIR_ALLOC, VIR_ALLOC_N, VIR_REALLOC_N, VIR_EXPAND_N)
(VIR_RESIZE_N, VIR_SHRINK_N, VIR_INSERT_ELEMENT)
(VIR_DELETE_ELEMENT, VIR_ALLOC_VAR, VIR_FREE): Document
which macros are safe in the presence of side effects.
* docs/hacking.html.in: Document this.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Use virBufferAddLit or virBufferAddChar instead.

VFIO device assignment requires a cgroup ACL to be setup for access to
the /dev/vfio/nn "group" device for any devices that will be assigned
to a guest. In the case of a host device that is allocated from a
pool, it was being allocated during qemuBuildCommandLine(), which is
called by qemuProcessStart() *after* the all-encompassing
qemuSetupCgroup() was called, meaning that the standard Cgroup ACL
setup wasn't creating ACLs for these devices allocated from pools.

One possible solution was to manually add a single ACL down inside
qemuBuildCommandLine() when networkAllocateActualDevice() is called,
but that has two problems: 1) the function that adds the cgroup ACL
requires a virDomainObjPtr, which isn't available in
qemuBuildCommandLine(), and 2) we really shouldn't be doing network
device setup inside qemuBuildCommandLine() anyway.

Instead, I've created a new function called
qemuNetworkPrepareDevices() which is called just before
qemuPrepareHostDevices() during qemuProcessStart() (explanation of
ordering in the comments), i.e. well before the call to
qemuSetupCgroup(). To minimize code churn in a patch that will be
backported to 1.0.5-maint, qemuNetworkPrepareDevices only does
networkAllocateActualDevice() and the bare amount of setup required
for type='hostdev network devices, but it eventually should do *all*
device setup for guest network devices.

Note that some of the code that was previously needed in
qemuBuildCommandLine() is no longer required when
networkAllocateActualDevice() is called earlier:

 * qemuAssignDeviceHostdevAlias() is already done further down in
   qemuProcessStart().

 * qemuPrepareHostdevPCIDevices() is called by
   qemuPrepareHostDevices() which is called after
   qemuNetworkPrepareDevices() in qemuProcessStart().

As hinted above, this new function should be moved into a separate
qemu_network.c (or similarly named) file along with
qemuPhysIfaceConnect(), qemuNetworkIfaceConnect(), and
qemuOpenVhostNet(), and expanded to call those functions as well, then
the nnets loop in qemuBuildCommandLine() should be reduced to only
build the commandline string (which itself can be in a separate
qemuInterfaceBuilldCommandLine() function as suggested by
Michal). However, this will require storing away an array of tapfd and
vhostfd that are needed for the commandline, so I would rather do that
in a separate patch and leave this patch at the minimum to fix the
bug.

If no todo.cfg is present, make sure the stub is in HTML5
format and clearly states that the config was not available
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The Windows port page currently links to pre-built libvirt
DLLs for release 0.8.8 which are 2 years old now. Until we
can reliably produce official Windows installers, point
people to the virt-viewer MSI installers instead which
include the libvirt DLLs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

On architectures not supporting the Intel specific programmable interval
timer, like e.g. S390, starting a domain with a clock definition containing
a pit timer results in the error "Option no-kvm-pit-reinjection not supported
for this target".

By moving the capability enablement for -no-kvm-pit-reinjection from the
InitQMPBasic section into the x86_64 and i686 only enablement section all
other architectures are no longer automatically enabled. In addition
architecture related capabilities enablements have refactored into a new
architecture bound capabilities initialization function.
Signed-off-by: NBoris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

We have seen an issue on s390x platform where domain XMLs larger than 1MB
were used. The define command was finished successfully. The dumpxml command
was not successful (i.e. could not encode message payload).

Enlarged message related sizes (e.g. maximum string size, message size, etc.)
to handle larger system configurations used on s390x platform.

To improve handling of the RPC message size the allocation during encode process
is changed to a dynamic one (i.e. starting with 64kB initial size and increasing
that size in steps up to 16MB if the payload data is larger).
Signed-off-by: NDaniel Hansel <daniel.hansel@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

The XSL for generating the API docs was missing the HTML5
namespace declarations. The todo and hvsupport scripts were
also missing the HTML5 doctype / namespace declaration.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When attempting to generate the native command line from an XML file
that uses graphics port auto allocation, the generated commandline
wouldn't be valid.

This patch adds fake autoallocation of ports as done when starting the
actual machine.

https://bugzilla.redhat.com/show_bug.cgi?id=924501 tracks a
problem that occurs if uid 107 is already in use at the time
libvirt is first installed.  In response that problem, Fedora
packaging guidelines were recently updated.  This fixes the
spec file to comply with the new guidelines:
https://fedoraproject.org/wiki/Packaging:UsersAndGroups

* libvirt.spec.in (daemon): Follow updated Fedora guidelines.
Signed-off-by: NEric Blake <eblake@redhat.com>

'make distcheck' complained:

ERROR: files left in build directory after distclean:
./python/libvirt.pyc
./tests/commandhelper.log

Problems introduced in commits f0154959 and 25ea8e47 (both v1.0.3).

* tests/commandtest.c (test21): Check (and clean) log file.
* tests/commanddata/test21.log: New file.
* python/Makefile.am (CLEANFILES): Clean up compiled python files.
Signed-off-by: NEric Blake <eblake@redhat.com>

On a mingw build, 'make distcheck' fails with:

  GEN      libvirt_qemu.def
make[3]: *** No rule to make target `../../src/libvirt_lxc.syms', needed by `libvirt_lxc.def'.  Stop.

I traced it to a missing entry in EXTRA_DIST.  But rather than keep
the entire list in sync, it is easier to list the three syms files
that drive .so files directly, and then reuse existing makefile
variables for the remaining files (that is, I validated that all
remaining files are added to SYM_FILES, possibly via USED_SYM_FILES,
according to makefile conditionals).

Problem introduced in commit 3d1596b0 (v1.0.2).

* src/Makefile.am (EXTRA_DIST): Ensure all syms files are shipped.
Signed-off-by: NEric Blake <eblake@redhat.com>

The code adaptation is not done right now, but in subsequent patches.
Hence I am not implementing syntax-check rule as it would break
compilation. Developers are strongly advised to use these new macros.
They are similar to VIR_ALLOC() logic: VIR_STRDUP(dst, src) returns zero
on success, -1 otherwise. In case you don't want to report OOM error,
use the _QUIET variant of a macro.

Add missing 'html:' namespace prefix to a few more XSL
rules for generating the table of contents links
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

I must have looked at this a couple dozen times before I noticed it
had "!=" instead of "==". Not doing this setup prevented qemu from
doing anything with the vfio group device.

The previous commit failed to update the XSL to take account
of fact that in XHTML mode the elements need namespace
prefixes. This caused every web page to be blank!

The rule generating the HTML docs passing the --html flag
to xsltproc. This makes it use the legacy HTML parser, which
either ignores or tries to fix all sorts of broken XML tags.
There's no reason why we should be writing broken XML in
the first place, so removing --html and adding the XHTML
doctype to all files forces us to create good XML.

This adds the XHTML doc type and fixes many, many XML tag
problems it exposes.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In the non linux case some callers like gather_scsi_host_caps needed the
return code of -1 while others like update_caps needed an empty
statement (to avoid a "statement without effect" warning). This is much
simpler solved by using a function instead of a define.

since it uses glibc's rpcgen.

A 'uri' parameter was added for the benefit of sanlock. This
causes a warning in the lockd driver though

2013-05-03 13:20:35.347+0000: 28403: error : virLockManagerLockDaemonNew:482 : internal error Unexpected parameter uri for object

Ignore this parameter, since lockd does not require it and it
is harmless if not used.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This should resolve https://bugzilla.redhat.com/show_bug.cgi?id=958907

Recent new addition of code to read/write active network state to the
NETWORK_STATE_DIR in the network driver broke startup for
qemu:///session. The network driver had several state file paths
hardcoded to /var, which could never possibly work in session mode.

This patch modifies *all* state files to use a variable string that is
set differently according to whether or not we're running
privileged. (It turns out that logDir was never used, so it's been
completely eliminated.)

There are very definitely other problems preventing dnsmasq and radvd
from running in non-privileged mode, but it's more consistent to have
the directories used by them be determined in the same fashion.

NB: I've noted before that the network driver is storing its state
(including dnsmasq and radvd state) in /var/lib, while qemu stores its
state in /var/run. It would probably have been better if the two
matched, but it's been this way for a long time, and changing it would
break running installations during an upgrade, so it's best to just
leave it as it is.

The QEMU migration code unconditionally sets the 'persistent'
cookie flag on the source host. The dest host, however, only
allows it during parsing if VIR_MIGRATE_PERSIST_DEST was
set. Make the source host only set it if this flag is
present.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The lockd plugin for the lock manager was not correctly
handling the release of resource locks. This meant that
during migration, or when pausing a VM, the locks would
not get released. This in turn made it impossible to
resume the domain, or finish migration

The F_DUPFD_CLOEXEC operation with fcntl() expects a single
int argument, specifying the minimum FD number for the newly
dup'd file descriptor. We were not specifying that causing
random stack data to be accessed as the FD number. Sometimes
that worked, sometimes it didn't.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

POSIX says pthread_t is opaque.  We can't guarantee if it is scaler
or a pointer, nor what size it is; and BSD differs from Linux.
We've also had reports of gcc complaining on attempts to cast it,
if we use a cast to the wrong type (for example, pointers have to be
cast to void* or intptr_t before being narrowed; while casting a
function return of scalar pthread_t to void* triggers a different
warning).

Give up on casts, and use unions to get at decent bits instead.  And
rather than futz around with figuring which 32 bits of a potentially
64-bit pointer are most likely to be unique, convert the rest of
the code base to use 64-bit values when using a debug id.

Based on a report by Guido Günther against kFreeBSD, but with a
fix that doesn't regress commit 4d970fd2 for FreeBSD.

* src/util/virthreadpthread.c (virThreadSelfID, virThreadID): Use
union to get at a decent bit representation of thread_t bits.
* src/util/virthread.h (virThreadSelfID, virThreadID): Alter
signature.
* src/util/virthreadwin32.c (virThreadSelfID, virThreadID):
Likewise.
* src/qemu/qemu_domain.h (qemuDomainJobObj): Alter type of owner.
* src/qemu/qemu_domain.c (qemuDomainObjTransferJob)
(qemuDomainObjSetJobPhase, qemuDomainObjReleaseAsyncJob)
(qemuDomainObjBeginNestedJob, qemuDomainObjBeginJobInternal): Fix
clients.
* src/util/virlog.c (virLogFormatString): Likewise.
* src/util/vireventpoll.c (virEventPollInterruptLocked):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 297c99a5 accidentally put a { where a ( belonged, causing make
rpm to fail.

Since the 'nparams' variable passed to virTypedParametersFree is
supposed to represent the size of the 'params' array, it is bad
practice to initialize it to a non-zero value, until the array
has been allocated.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If an early dispatch check caused a jump to the 'cleanup' branch
then virTypeParamsFree() would be called with an uninitialized
'nparams' variable. Fortunately 'params' is initialized to NULL,
so the uninitialized 'nparams' variable would not be used.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The use of () in a simple boolean comparison was not
required
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The call to virReportError conditionally switched between
two format strings, with different numbers of placeholders.
This meant the format string with no placeholders was not
protected by a "%s".
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit 297c99a5 says it works from 1.0.5 but it only got pushed
after the 1.0.5 release.

Commit 7c9a2d88 missed inclusion of virstring.h in a few places
when WITH_HAL is defined, causing build failures.

More fallout from commit 7c9a2d88 dropping too many headers.  Fixes:

In file included from ../../src/vbox/vbox_glue.c:26:0:
../../src/vbox/vbox_MSCOMGlue.c: In function 'vboxLookupVersionInRegistry':
../../src/vbox/vbox_MSCOMGlue.c:435:5: error: implicit declaration of function 'virParseVersionString' [-Werror=implicit-function-declaration]
...
../../src/vbox/vbox_driver.c: In function 'vboxConnectOpen':
../../src/vbox/vbox_driver.c:147:5: error: implicit declaration of function 'getuid' [-Werror=implicit-function-declaration]
../../src/vbox/vbox_driver.c:147:5: error: nested extern declaration of 'getuid' [-Werror=nested-externs]

* src/vbox/vbox_MSCOMGlue.c (includes): Add missing includes.
* src/vbox/vbox_driver.c (includes): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 776d49f4 added a static function that is only called
conditionally; leading to this compile error on mingw:

  CC       libvirt_util_la-virprocess.lo
../../src/util/virprocess.c:624:26: error: 'struct rlimit' declared inside parameter list [-Werror]
../../src/util/virprocess.c:624:26: error: its scope is only this definition or declaration, which is probably not what you want [-Werror]
../../src/util/virprocess.c:622:1: error: 'virProcessPrLimit' defined but not used [-Werror=unused-function]

* src/util/virprocess.c (virProcessPrLimit): Only declare
virProcessPrLimit when used.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 7c9a2d88 cleaned up too many headers; FreeBSD builds
failed due to:

util/virutil.c:556: warning: implicit declaration of function 'canonicalize_file_name'

(Not sure which Linux header leaked this declaration, but gnulib
only guarantees it in stdlib.h)

libvirt.c:956: warning: implicit declaration of function 'virGetUserConfigDirectory'

(Here, a build on Linux was picking up virutil.h indirectly via
one of the conditional driver headers, where that driver was not
being built on my FreeBSD setup)

* src/util/virutil.c (includes): Need <stdlib.h> for
canonicalize_file_name.
* src/libvirt.c (includes): Use "virutil.h" unconditionally,
rather than relying on conditional indirect inclusion.
Signed-off-by: NEric Blake <eblake@redhat.com>

virAsprintf(&foo, "%s", bar) is wasteful compared to
foo = strdup(bar) (or eventually, VIR_STRDUP(foo, bar),
but one thing at a time...).

Noticed while reviewing Laine's attempt to clean up broken
qemu:///session.

* cfg.mk (sc_prohibit_asprintf): Enhance rule.
* src/esx/esx_storage_backend_vmfs.c
(esxStorageBackendVMFSVolumeLookupByKey): Fix offender.
* src/network/bridge_driver.c (networkStateInitialize): Likewise.
* src/nwfilter/nwfilter_dhcpsnoop.c (virNWFilterSnoopDHCPOpen):
Likewise.
* src/storage/storage_backend_sheepdog.c
(virStorageBackendSheepdogRefreshVol): Likewise.
* src/util/vircgroup.c (virCgroupAddTaskStrController): Likewise.
* src/util/virdnsmasq.c (addnhostsAdd): Likewise.
* src/xen/block_stats.c (xenLinuxDomainDeviceID): Likewise.
* src/xen/xen_driver.c (xenUnifiedConnectOpen): Likewise.
* tools/virsh.c (vshGetTypedParamValue): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

The libvirt 1.0.5 tarball is missing a file which renders 'make
check' broken; first reported on list by Guido Günther.

* python/Makefile.am (EXTRA_DIST): Hoist sanitytest.py out of
HAVE_PYTHON conditional.
Signed-off-by: NEric Blake <eblake@redhat.com>