The Mingw32 linker highlighted that the symbols for virtime.h
declared in libvirt_private.syms were incorrect

* src/libvirt_private.syms: Fix virtime.h symbols

When QEMU guest finishes its shutdown sequence, qemu stops virtual CPUs
and when started with -no-shutdown waits for us to kill it using
SGITERM. Since QEMU is flushing its internal buffers, some time may pass
before QEMU actually dies. We mistakenly used "paused" state (and
events) for this which is quite confusing since users may see a domain
going to pause while they expect it to shutdown. Since we already have
"shutdown" state with "the domain is being shut down" semantics, we
should use it for this state.

However, the state didn't have a corresponding event so I created one
and called its detail as VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED (guest OS
finished its shutdown sequence) with the intent to add
VIR_DOMAIN_EVENT_SHUTDOWN_STARTED in the future if we have a
sufficiently capable guest agent that can notify us when guest OS starts
to shutdown.

Otherwise connections to older libvirt abort with:

$ virsh -c qemu+ssh://host.example.com/system list
error: invalid connection pointer in virDrvSupportsFeature
error: failed to connect to the hypervisor

Tested against 0.8.3 and 0.9.8-rc2.

This patch adds binding for virNodeGetMemoryStats method of libvirtd.
Return value is represented as a python dictionary mapping field
names to values.

This patch adds binding for virNodeGetCPUStats method of libvirtd.
Return value is represented as a python dictionary mapping field names
to values.

https://bugzilla.redhat.com/show_bug.cgi?id=648855 mentioned a
misuse of 'an' where 'a' is proper; that has since been fixed,
but a search found other problems (some were a spelling error for
'and', while most were fixed by 'a').

* daemon/stream.c: Fix grammar.
* src/conf/domain_conf.c: Likewise.
* src/conf/domain_event.c: Likewise.
* src/esx/esx_driver.c: Likewise.
* src/esx/esx_vi.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* src/rpc/virnetserverprogram.c: Likewise.
* src/storage/storage_backend_fs.c: Likewise.
* src/util/conf.c: Likewise.
* src/util/dnsmasq.c: Likewise.
* src/util/iptables.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
* src/xen/xend_internal.c: Likewise.
* src/xen/xs_internal.c: Likewise.
* tools/virsh.c: Likewise.

We want our tarballs to be complete - this means that any
generated file that gets shipped as part of the tarball so that
ordinary users don't have to rebuild it must be something
that the maintainer can generate.  There have been various
reports of random build failures when using libvirt.git
instead of a tarball, and often it is due to missing a
maintainer-specific tool to produce one of these generated
files.  This patch raises the bar for what you must have
installed to build libvirt.git, but does not impact what
you can get away with for building tarballs.

Note: It still remains possible to do a successful 'make dist'
without these tools, when starting from a release tarball.

* bootstrap.conf (buildreq): Add tools that maintainers need for a
successful 'make dist' from a fresh git checkout.

virBufferContentAndReset (intentionally) returns NULL for a buffer
with no content, but it is feasible to invoke a command with an
explicit empty string.

* src/util/command.c (virCommandAddEnvBuffer): Reject empty string.
(virCommandAddArgBuffer): Allow explicit empty argument.
* tests/commandtest.c (test9): Test it.
* tests/commanddata/test9.log: Adjust.

The RPC fixups needed on Linux are also needed on cygwin, and
worked without further tweaking to the list of fixups.  Also,
unlike BSD, Cygwin exports 'struct ifreq', but unlike Linux,
Cygwin lacks the ioctls that we were using 'struct ifreq' to
access.  This patch allows compilation under cygwin.

* src/rpc/genprotocol.pl: Also perform fixups on cygwin.
* src/util/virnetdev.c (HAVE_STRUCT_IFREQ): Also require AF_PACKET
definition.
* src/util/virnetdevbridge.c (virNetDevSetupControlFull): Only
compile if SIOCBRADDBR works.

I had previously tested commit 059d746d with -O intentionally omitted
from my CFLAGS; but that means that I missed out on this warning
from gcc 4.6.2 when optimizations are enabled:

util/buf.c: In function 'virBufferGetIndent':
util/buf.c:86:1: error: function might be candidate for attribute 'pure' [-Werror=suggest-attribute=pure]

While it is probably a good idea to add the attributes and silence
this warning, it's also invasive; 'make -k' found more than 75 such
complaints.  And it doesn't help that gcc 4.6.2 is still buggy
(coreutils reported a case where gcc 4.6.2 incorrectly suggested
marking a function pure that incremented a global variable; fixed
in gcc 4.7).  So the best fix for now is to disable the warning.

It also doesn't help that I stumbled across another problem - gcc
documents that -Wsuggest-attribute=pure only warns if you use -O,
or if you use -fipa-pure-const.  But in practice, when I omitted -O
but added -fipa-pure-const, the warnings are fickle - I got warnings
for simple compilation that disappeared when I also added -fPIC.
And the way libtool compiles things is with -fPIC first, then without
-fPIC but with errors sent to /dev/null - which meant that without
disabling -Wsuggest-attribute=pure, I got a compile error with no
message.  :(  See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10197

* m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Silence
-Wsuggest-attribute warnings for now.

The pathname for the pipe for tunnelled migration is unresolvable.  The
libvirt apparmor driver therefore refuses access, causing migration to
fail.  If we can't resolve the path, the worst that can happen is that
we should have given permission to the file but didn't.  Otherwise
(especially since this is a /proc/$$/fd/N file) the file is already open
and libvirt won't be refused access by apparmor anyway.

Also adjust virt-aa-helper to allow access to the
*.tunnelmigrate.dest.name files.

For more information, see https://launchpad.net/bugs/869553.
Signed-off-by: NSerge Hallyn <serge.hallyn@canonical.com>

Originaly, the code checked if another client is the queue and infered
ownership of the buck from that. Commit fa959500
added a separate variable to track the buck. That caused, that a new
call might enter claiming it has the buck, while another thread was
signalled to take the buck. This ends in two threads claiming they hold
the buck and entering poll(). This happens due to a race on waking up
threads on the client lock mutex.

This caused multi-threaded clients to hang, most prominently visible and
reproducible on python based clients, like virt-manager.

This patch causes threads, that have been signalled to take the buck to
re-check if buck is held by another thread.

With fragments borrowed from David Steven's previous submission and some
further modifications:

A set of modifications to filters to handle multiple IP addresses
(and MAC addresses) per interface.

Also:
- enable DHCP traffic from VM to any DHCP server
- will require an update to a libvirt-tck data file
Signed-off-by: NDavid L Stevens <dlstevens@us.ibm.com>
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>

Reported by kato.tomoyuki@jp.fujitsu.com at
https://bugzilla.redhat.com/show_bug.cgi?id=749564

* tools/virsh.c (info_network_info): Mark string for translation.

Many of these were mentioned by Yuri Chornoivan in:
https://bugzilla.redhat.com/show_bug.cgi?id=669506

* src/esx/esx_vi.c (esxVI_WaitForTaskCompletion): Fix spelling.
* src/conf/netdev_vport_profile_conf.c
(virNetDevVPortProfileParse): Likewise.
* src/xen/xend_internal.c (xenDaemonDomainSetVcpusFlags):
Likewise.
* src/xen/xm_internal.c (xenXMDomainSetVcpusFlags): Likewise.
* src/esx/esx_util.c (esxUtil_ResolveHostname): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemBuild): Likewise.
* daemon/libvirtd.conf: Likewise.
* src/util/logging.c (virLogMessage): Likewise.
* src/uml/uml_conf.c (umlBuildCommandLineNet): Likewise.
* src/vmx/vmx.c (virVMXFormatEthernet): Likewise.

* .gnulib: Update to latest, for improved 'make syntax-check' and
compiler warnings.
* m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS):
Re-silence -Wformat-nonliteral.
* cfg.mk (_test_script_regex): Recognize our test scripts.
* gnulib/local/lib/*.diff: Drop, now that gnulib has this.
* tests/virsh-optparse: Fix use of compare.
* tests/virsh-schedinfo: Likewise.

I got this failure on FreeBSD:

shunloadtest.c: In function 'main':
shunloadtest.c:150: error: 'EXIT_AM_SKIP' undeclared (first use in this function)

but inspection showed several other problems, all fixed here.

* tests/domainsnapshotxml2xmltest.c [!WITH_QEMU]: Ensure
EXIT_AM_SKIP is defined.
* tests/esxutilstest.c [!WITH_ESX]: Likewise.
* tests/openvzutilstest.c [!WITH_OPENVZ]: Likewise.
* tests/qemuargv2xmltest.c [!WITH_QEMU]: Likewise.
* tests/qemuhelptest.c [!WITH_QEMU]: Likewise.
* tests/qemuxml2argvtest.c [!WITH_QEMU]: Likewise.
* tests/qemuxml2xmltest.c [!WITH_QEMU]: Likewise.
* tests/qemuxmlnstest.c [!WITH_QEMU]: Likewise.
* tests/shunloadtest.c [!linux]: Likewise.
* tests/vmx2xmltest.c [!WITH_VMX]: Likewise.
* tests/xml2vmxtest.c [!WITH_VMX]: Likewise.

For unknown reasons, the shunloadtest will crash on Fedora 16
inside dlopen()

 (gdb) bt
 #0  0x00000000000050e6 in ?? ()
 #1  0x00007ff61a77b9d5 in floor () from /lib64/libm.so.6
 #2  0x00007ff61e522963 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2
 #3  0x00007ff61e5297e6 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
 #4  0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
 #5  0x00007ff61e52917a in _dl_open () from /lib64/ld-linux-x86-64.so.2
 #6  0x00007ff61e0f6f26 in dlopen_doit () from /lib64/libdl.so.2
 #7  0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
 #8  0x00007ff61e0f752f in _dlerror_run () from /lib64/libdl.so.2
 #9  0x00007ff61e0f6fc1 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2
 #10 0x0000000000400a15 in main (argc=<optimized out>, argv=<optimized out>) at shunloadtest.c:105

Changing from RTLD_NOW to RTLD_LAZY avoids this problem,
but quite possibly does not fix the root cause.

* shunloadtest.c: s/NOW/LAZY/

This ought to fix the build if you have net/if.h but do
not have struct ifreq

* configure.ac: Check for struct ifreq in net/if.h
* src/util/virnetdev.c: Conditionalize to avoid use of
  struct ifreq if it does not exist

probes.h can only be generated on Linux, and then only with dtrace
installed.  If it is part of the tarball, then either 'make dist'
will fail if you don't have that setup, or we would have to start
keeping probes.h in libvirt.git.  Since we only need it to be
generated when dtrace is in use, it's better to avoid shipping
it in the first place, and avoid tracking it in git.

Meanwhile, there is a build dependency - since the RPC code is
generated, it can be built early; but when dtrace is enabled, we
must ensure probes.h is built even earlier.  Commit 1afcfbdd tried
to fix this, but did so in a way that added probes.h into the
tarball, and broke VPATH as well.  Commit ecbca767 fixed VPATH,
but didn't fix the more fundamental problem.  This patch solves
the issue by adding a dependency instead.

Tested with 'make dist' in a clean VPATH builds, for both
'./configure --without-dtrace' and './configure --with-dtrace';
all configurations were able to correctly build a tarball, and
the dtrace configuration no longer sticks probes.h in the tarball.

* src/Makefile.am (REMOTE_DRIVER_GENERATED): Don't ship probes.h;
rather, make it a dependency.

Fix a logic error, the initial value of ret = -1, if just set --config,
it will goto endjob directly without doing its really job here.
Signed-off-by: NLei Li <lilei@linux.vnet.ibm.com>

The glibc time.h header has an undocumented __isleap macro
that we are using. Since it is undocumented & does not appear
on any other OS, stop using it and just define the macro in
libvirt code instead.

* src/util/virtime.c: Remove __isleap usage

Currently virsh supports only ^] as escape character for console.
However, some users might want to use something else. This patch
creates such ability by specifying '-e' switch on virsh command
line.

Only one IPv4 DHCP definition is supported. Originally the code checked
for a multiple definition and returned an error, but the new domain
definition was already added to networks. This patch moves the check
before the newly defined network is added to active networks.

 *src/network/bridge_driver.c: networkDefine(): - move multiple IPv4
                                                  addresses check before
                                                  definition is used.

We have several directories that are created on the fly, and which
only contain state relevant to a running libvirtd process (all
located in /var/run).  Since the directories are created as needed,
and make no sense without a running libvirtd, we want them deleted
if libvirt is uninstalled.  And in F15 and newer, /var/run is on
tmpfs (forcing us to recreate on the fly); which means that someone
trying to verify a complete rpm will fail if the directory does not
currently exist because libvirtd has not been started since boot.
The solution, then, is to mark the directories as %ghost, so that
rpm knows that we own them and will clean it up if libvirt is
uninstalled, but will no longer create the directory for us at
install, nor complain at verify time if the directory does not exist.

See https://bugzilla.redhat.com/show_bug.cgi?id=656611.

* libvirt.spec.in (%files): Add %ghost to temporary directories
that we don't install, but want cleaned up on libvirt removal.

Not only was ctl->quit accessed without a mutex but unfortunately,
virEventAddTimeout only interrupts the poll when event loop is running
so the hack needs to add a timeout that will make next poll return
immediately without blocking.

Detected by Coverity. Leak introduced in commit 90074ecf.
Signed-off-by: NAlex Jia <ajia@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Detected by Coverity. Leak introduced in commit c1df2c14.

Two bugs here:
1. memory leak on successful parse
2. failure to parse still returned success
Signed-off-by: NAlex Jia <ajia@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Detected by Coverity.  Leak introduced in commit 8866eed0.

Two bugs here:
1. logfd wasn't closed on all return paths
2. if we failed to mark a domain autodestroy, then the domain
was not made transient but we still returned success
Signed-off-by: NAlex Jia <ajia@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Detected by Coverity. Leak introduced in commit 673adba5.

Two separate bugs here:
1. call was not freed on all error paths
2. virCondDestroy was called even if virCondInit failed
Signed-off-by: NAlex Jia <ajia@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Detected by Coverity. Leak introduced in commit 0873b688.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Detected by Coverity. Leak introduced in commit 109efd79.
Signed-off-by: NAlex Jia <ajia@redhat.com>

To add support for running libvirt on PowerPC, a CPU driver for the
PowerPC platform must be added.
Most generic cpu driver routines such as CPU compare, decode, etc
are based on CPUID comparison and are not relevant for non-x86
platforms.
Here, we introduce stubs for relevant PowerPC routines invoked by libvirt.
Signed-off-by: NPrerna Saxena <prerna@linux.vnet.ibm.com>
Signed-off-by: NAnton Blanchard <anton@au.ibm.com>

filter 0-device-weight when:

  - getting blkio parameters with --config
  - starting up a domain

When testing with blkio, I found these issues:

  (dom is down)
  virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,500
  virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,0
  virsh blkiotune dom
  weight         : 800
  device_weight  : /dev/sda,200,/dev/sdb,0

  # issue 1: shows 0 device weight of /dev/sdb that may confuse user

  (continued)
  virsh start dom

  # issue 2: If /dev/sdb doesn't exist, libvirt refuses to bring the
  # dom up because it wants to set the device weight to 0 of a
  # non-existing device. Since 0 means no weight-limit, we really don't
  # have to set it.

Prior to this patch, for a running dom, the commands:

$ virsh blkiotune dom --device-weights /dev/sda,502,/dev/sdb,498
$ virsh blkiotune dom --device-weights /dev/sda,503
$ virsh blkiotune dom
weight         : 500
device_weight  : /dev/sda,503

claim that /dev/sdb no longer has a non-default weight, but
directly querying cgroups says otherwise:

$ cat /cgroup/blkio/libvirt/qemu/dom/blkio.weight_device
8:0     503
8:16    498

After this patch, an explicit 0 is required to remove a device path
from the XML, and omitting a device path that was previously
specified leaves that device path untouched in the XML, to match
cgroups behavior.

* src/qemu/qemu_driver.c (parseBlkioWeightDeviceStr): Rename...
(qemuDomainParseDeviceWeightStr): ...and use correct type.
(qemuDomainSetBlkioParameters): After parsing string, modify
rather than replacing existing table.
* tools/virsh.pod (blkiotune): Tweak wording.

The next patch will make it possible to have virDomainSetBlkioParameters
leave device weights unchanged if they are not mentioned in the incoming
string, but this only works if the list of block weights does not allow
duplicate paths.  Technically, a user can still confuse libvirt by
passing alternate spellings that resolve to the same device, but it
is not worth worrying about working around that kind of abuse.

* src/conf/domain_conf.c (virDomainDefParseXML): Require unique
paths.

When setting both blkio weight and device-weights at the same time, the weight
is lost. Fix it.

Signed-off-by: NLei Li <lilei@linux.vnet.ibm.com>
Signed-off-by: NZhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>