Earlier, when the number of vcpus was greater than the topology allowed,
libvirt didn't raise an error and continued, resulting in running qemu
with parameters making no sense. Even though qemu did not report any
error itself, the number of vcpus was set to maximum allowed by the
topology.

Pick up recent gnulib improvements.

* .gnulib: Update to latest.
* bootstrap: Resync.
* bootstrap.conf (gnulib_tool_option_extras): Adjust to bootstrap
changes.
* gnulib/lib/Makefile.am: Likewise.

Detected by Coverity.  Although unlikely, if we are ever started
with stdin closed, we could reach a situation where we open a
uuid file but then fail to close it, making that file the new
stdin for the rest of the process.

* src/util/uuid.c (getDMISystemUUID): Allow for stdin.

Commit 69f0b446 failed to update the expected test output.

* tests/virshtest.c (testCompareListDefault)
(testCompareListCustom): Adjust to recent code change.

Currently the LXC controller attempts to deal with EOF on a
tty by spawning a thread to do an edge triggered epoll_wait().
This avoids the normal event loop spinning on POLLHUP. There
is a subtle mistake though - even after seeing POLLHUP on a
master PTY, it is still perfectly possible & valid to write
data to the PTY. There is a buffer that can be filled with
data, even when no client is present.

The second mistake is that the epoll_wait() thread was not
looking for the EPOLLOUT condition, so when a new client
connects to the LXC console, it had to explicitly send a
character before any queued output would appear.

Finally, there was in fact no need to spawn a new thread to
deal with epoll_wait(). The epoll file descriptor itself
can be poll()'d on normally.

This patch attempts to deal with all these problems.

 - The blocking epoll_wait() thread is replaced by a poll
   on the epoll file descriptor which then does a non-blocking
   epoll_wait() to handle events
 - Even if POLLHUP is seen, we continue trying to write
   any pending output until getting EAGAIN from write.
 - Once write returns EAGAIN, we modify the epoll event
   mask to also look for EPOLLOUT

* src/lxc/lxc_controller.c: Avoid stalled I/O upon
  connected to an LXC console

Domain IDs are at least 16 bits for most hypervisors, theoretically
event 32-bits.  3 characters is clearly too small an alignment.
Increase alignment to 5 characters to allow 16-bit domain IDs to
display cleanly. Commonly seen with LXC where domain IDs are the
process IDs by default.  Also increase the 'name' field from 20
to 30 characters to cope with longer guest names which are quite
common

If client stream does not have any data to sink and neither received
EOF, a dummy packet is sent to the daemon signalising client is ready to
sink some data. However, after we added event loop to client a race may
occur:

Thread 1 calls virNetClientStreamRecvPacket and since no data are cached
nor stream has EOF, it decides to send dummy packet to server which will
sent some data in turn. However, during this decision and actual message
exchange with server -

Thread 2 receives last stream data from server. Therefore an EOF is set
on stream and if there is a call waiting (which is not yet) it is woken
up. However, Thread 1 haven't sent anything so far, so there is no call
to be woken up. So this thread sent dummy packet to daemon, which
ignores that as no stream is associated with such packet and therefore
no reply will ever come.

This race causes client to hang indefinitely.

Just like command "domblklist", the command extracts "type",
"source", "target", "model", and "MAC" of all virtual interfaces
from domain XML (live or persistent).

QEMU does not support security_model for anything but 'path' fs driver type.
Currently in libvirt, when security_model ( accessmode attribute) is not
specified it auto-generates it irrespective of the fs driver type, which
can result in a qemu error for drivers other than path. This patch ensures
that the qemu cmdline is correctly generated by taking into account the
fs driver type.
Signed-off-by: NDeepak C Shetty <deepakcs@linux.vnet.ibm.com>

The above option helps to differentiate between implicit and explicit
interface pools.

If a system has 64 or more VF's, it is quite tedious to mention each VF
in the interface pool.
The following modification will implicitly create an interface pool from
the SR-IOV PF.

This element will help the user to just specify the SR-IOV physical
function in order to access all the Virtual functions attached to it.

This functions enables us to get the Virtual Functions attached to
a Physical function given the name of a SR-IOV physical functio.

In order to accomplish the task, added a getter function pciGetDeviceAddrString
to get the BDF of the Virtual Function in a char array.

Although the netcf interface driver can in theory be used by
the stateless drivers, in practice none of them want to use
it because they have different ways of dealing with interfaces.

Furthermore, if you have mingw32-netcf installed, then the
libvirt mingw32 build will fail with

  ../../src/interface/netcf_driver.c:644:5: error: unknown field 'close_used_without_including_unistd_h' specified in initializer

 * configure.ac: disable netcf if built without libvirtd

The autobuilder pointed out an odd failure on mingw:
../../src/interface/netcf_driver.c:644:5: error: unknown field 'close_used_without_including_unistd_h' specified in initializer
cc1: warnings being treated as errors

This is because the gnulib headers #define close to different strings,
according to which headers are included, in order to work around some
odd mingw problems with close(), and these defines happen to also
affect field members declared with a name of struct foo.close. As long
as all headers are included before both the definition and use of the
struct, the various #define doesn't matter, but the netcf file hit
an instance where things were included in a different order.  Fix this
for all clients that use a struct member named 'close'.

* src/driver.h: Include <unistd.h> before using 'close'.

For some weird reason, i686-pc-mingw32-gcc version 4.6.1 at -O2 complained:
../../src/conf/nwfilter_params.c: In function 'virNWFilterVarCombIterCreate':
../../src/conf/nwfilter_params.c:346:23: error: 'minValue' may be used uninitialized in this function [-Werror=uninitialized]
../../src/conf/nwfilter_params.c:319:28: note: 'minValue' was declared here
../../src/conf/nwfilter_params.c:344:23: error: 'maxValue' may be used uninitialized in this function [-Werror=uninitialized]
../../src/conf/nwfilter_params.c:319:18: note: 'maxValue' was declared here
cc1: all warnings being treated as errors

even though all paths of the preceding switch statement either
assign the variables or return.

* src/conf/nwfilter_params.c (virNWFilterVarCombIterAddVariable):
Initialize variables.

This patch adds documentation about the new 'ways' that users can
access the contents of variables in filters:

- access via index: $TEST[2]
- access via iterators $TEST[@1]

This patch adds a couple of XML parser / schema validator test cases
for the new 'ways' to access variables via index or iterator.

Address side effect of accessing a variable via an index: Filters
accessing a variable where an element is accessed that is beyond the
size of the list (for example $TEST[10] and only 2 elements are available)
cannot instantiate that filter. Test for this and report proper error
to user.

This patch adds access to single elements of variables via index. Example:

  <rule action='accept' direction='in' priority='500'>
    <tcp srcipaddr='$ADDR[1]' srcportstart='$B[2]'/>
  </rule>

This patch introduces the capability to use a different iterator per
variable.

The currently supported notation of variables in a filtering rule like

  <rule action='accept' direction='out'>
     <tcp  srcipaddr='$A' srcportstart='$B'/>
  </rule>

processes the two lists 'A' and 'B' in parallel. This means that A and B
must have the same number of 'N' elements and that 'N' rules will be 
instantiated (assuming all tuples from A and B are unique).

In this patch we now introduce the assignment of variables to different
iterators. Therefore a rule like

  <rule action='accept' direction='out'>
     <tcp  srcipaddr='$A[@1]' srcportstart='$B[@2]'/>
  </rule>

will now create every combination of elements in A with elements in B since
A has been assigned to an iterator with Id '1' and B has been assigned to an
iterator with Id '2', thus processing their value independently.

The first rule has an equivalent notation of

  <rule action='accept' direction='out'>
     <tcp  srcipaddr='$A[@0]' srcportstart='$B[@0]'/>
  </rule>

In this patch we introduce testing whether the iterator points to a
unique set of entries that have not been seen before at one of the previous
iterations. The point is to eliminate duplicates and with that unnecessary
filtering rules by preventing identical filtering rules from being
instantiated.
Example with two lists:

list1 = [1,2,1]
list2 = [1,3,1]

The 1st iteration would take the 1st items of each list -> 1,1
The 2nd iteration would take the 2nd items of each list -> 2,3
The 3rd iteration would take the 3rd items of each list -> 1,1 but
skip them since this same pair has already been encountered in the 1st
iteration

Implementation-wise this is solved by taking the n-th element of list1 and
comparing it against elements 1..n-1. If no equivalent is found, then there
is no possibility of this being a duplicate. In case an equivalent element
is found at position i, then the n-th element in the 2nd list is compared
against the i-th element in the 2nd list and if that is not the same, then
this is a unique pair, otherwise it is not unique and we may need to do
the same comparison on the 3rd list.

When running virsh migrate with --xml option and actual xml file doesn't
exist, virsh hasn't output any error information, although return value
is 1.

* tools/virsh.c: Raising a appropriate error information when operation fails.

* How to reproduce?
% virsh migrate <domain> --live qemu+ssh://<target host>/system --xml non-existent.xml
% echo $?

* Fixed result:
error: file 'non-existent.xml' doesn't exist
Signed-off-by: NAlex Jia <ajia@redhat.com>

When sVirt is integrated with the LXC driver, it will be neccessary
to invoke the security driver APIs using only a virDomainDefPtr
since the lxc_container.c code has no virDomainObjPtr available.
Aside from two functions which want obj->pid, every bit of the
security driver code only touches obj->def. So we don't need to
pass a virDomainObjPtr into the security drivers, a virDomainDefPtr
is sufficient. Two functions also gain a 'pid_t pid' argument.

* src/qemu/qemu_driver.c, src/qemu/qemu_hotplug.c,
  src/qemu/qemu_migration.c, src/qemu/qemu_process.c,
  src/security/security_apparmor.c,
  src/security/security_dac.c,
  src/security/security_driver.h,
  src/security/security_manager.c,
  src/security/security_manager.h,
  src/security/security_nop.c,
  src/security/security_selinux.c,
  src/security/security_stack.c: Change all security APIs to use a
  virDomainDefPtr instead of virDomainObjPtr

When disk snapshots were first implemented, libvirt blindly refused
to allow an external snapshot destination that already exists, since
qemu will blindly overwrite the contents of that file during the
snapshot_blkdev monitor command, and we don't like a default of
data loss by default.  But VDSM has a scenario where NFS permissions
are intentionally set so that the destination file can only be
created by the management machine, and not the machine where the
guest is running, so that libvirt will necessarily see the destination
file already existing; adding a flag will allow VDSM to force the file
reuse without libvirt complaining of possible data loss.

https://bugzilla.redhat.com/show_bug.cgi?id=767104

* include/libvirt/libvirt.h.in (virDomainSnapshotCreateFlags): Add
VIR_DOMAIN_SNAPSHOT_CREATE_REUSE_EXT.
* src/libvirt.c (virDomainSnapshotCreateXML): Document it.  Add
note about partial failure.
* tools/virsh.c (cmdSnapshotCreate, cmdSnapshotCreateAs): Add new
flag.
* tools/virsh.pod (snapshot-create, snapshot-create-as): Document
it.
* src/qemu/qemu_driver.c (qemuDomainSnapshotDiskPrepare)
(qemuDomainSnapshotCreateXML): Implement the new flag.

We had loads of different styles in describing the @flags parameter
for various APIs, as well as several APIs that didn't list which
enums provided the bit values valid for the flags.

The end result is one of two formats:
@flags: bitwise-OR of vir...Flags
@flags: extra flags; not used yet, so callers should always pass 0

* src/libvirt.c: Use common sentences for flags.  Also,
(virDomainGetBlockIoTune): Mention virTypedParameterFlags.
(virConnectOpenAuth): Mention virConnectFlags.
(virDomainMigrate, virDomainMigrate2, virDomainMigrateToURI)
(virDomainMigrateToURI2): Mention virDomainMigrateFlags.
(virDomainMemoryPeek): Mention virDomainMemoryFlags.
(virStoragePoolBuild): Mention virStoragePoolBuildFlags.
(virStoragePoolDelete): Mention virStoragePoolDeleteFlags.
(virStreamNew): Mention virStreamFlags.
(virDomainOpenGraphics): Mention virDomainOpenGraphicsFlags.

This *kind of* addresses:

  https://bugzilla.redhat.com/show_bug.cgi?id=772395

(it doesn't eliminate the failure to start, but causes libvirt to give
a better idea about the cause of the failure).

If a guest uses a kvm emulator (e.g. /usr/bin/qemu-kvm) and the guest
is started when kvm isn't available (either because virtualization is
unavailable / has been disabled in the BIOS, or the kvm modules
haven't been loaded for some reason), a semi-cryptic error message is
logged:

  libvirtError: internal error Child process (LC_ALL=C
  PATH=/sbin:/usr/sbin:/bin:/usr/bin /usr/bin/qemu-kvm -device ? -device
  pci-assign,? -device virtio-blk-pci,? -device virtio-net-pci,?) status
  unexpected: exit status 1

This patch notices at process start that a guest needs kvm, and checks
for the presence of /dev/kvm (a reasonable indicator that kvm is
available) before trying to execute the qemu binary. If kvm isn't
available, a more useful (too verbose??) error is logged.

It should be a copy-paste error, the result is programming will result in an
infinite loop again due to without iterating 'j' variable.

* src/qemu/qemu_driver.c: fix a typo on qemuDomainSetBlkioParameters.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=770520Signed-off-by: NAlex Jia <ajia@redhat.com>

This is actually a patch for VM Manager for Android.

https://www.redhat.com/archives/libvirt-users/2011-November/msg00076.html

I previously mentioned [1] a PolicyKit issue where libvirt would
proceed with authentication even though polkit-auth failed:

testusr xen134:~> virsh list --all
Attempting to obtain authorization for org.libvirt.unix.manage.
polkit-grant-helper: given auth type (8 -> yes) is bogus
Failed to obtain authorization for org.libvirt.unix.manage.
 Id Name                 State
----------------------------------
  0 Domain-0             running
  - sles11sp1-pv         shut off

AFAICT, libvirt attempts to obtain a privilege it already has,
causing polkit-auth to fail with above message.  Instead of calling
obtain and then checking auth, IMO the workflow should be for the
server to check auth first, and if that fails ask the client to
obtain it and check again.  This workflow also allows for checking
only successful exit of polkit-auth in virConnectAuthGainPolkit().

[1] https://www.redhat.com/archives/libvir-list/2011-December/msg00837.html

In the past, generic SCSI commands issued from a guest to a virtio
disk were always passed through to the underlying disk by qemu, and
the kernel would also pass them on.

As a result of CVE-2011-4127 (see:
http://seclists.org/oss-sec/2011/q4/536), qemu now honors its
scsi=on|off device option for virtio-blk-pci (which enables/disables
passthrough of generic SCSI commands), and the kernel will only allow
the commands for physical devices (not for partitions or logical
volumes). The default behavior of qemu is still to allow sending
generic SCSI commands to physical disks that are presented to a guest
as virtio-blk-pci devices, but libvirt prefers to disable those
commands in the standard virtio block devices, enabling it only when
specifically requested (hopefully indicating that the requester
understands what they're asking for). For this purpose, a new libvirt
disk device type (device='lun') has been created.

device='lun' is identical to the default device='disk', except that:

1) It is only allowed if bus='virtio', type='block', and the qemu
   version is "new enough" to support it ("new enough" == qemu 0.11 or
   better), otherwise the domain will fail to start and a
   CONFIG_UNSUPPORTED error will be logged).

2) The option "scsi=on" will be added to the -device arg to allow
   SG_IO commands (if device !='lun', "scsi=off" will be added to the
   -device arg so that SG_IO commands are specifically forbidden).

Guests which continue to use disk device='disk' (the default) will no
longer be able to use SG_IO commands on the disk; those that have
their disk device changed to device='lun' will still be able to use SG_IO
commands.

*docs/formatdomain.html.in - document the new device attribute value.
*docs/schemas/domaincommon.rng - allow it in the RNG
*tests/* - update the args of several existing tests to add scsi=off, and
 add one new test that will test scsi=on.
*src/conf/domain_conf.c - update domain XML parser and formatter

*src/qemu/qemu_(command|driver|hotplug).c - treat
 VIR_DOMAIN_DISK_DEVICE_LUN *almost* identically to
 VIR_DOMAIN_DISK_DEVICE_DISK, except as indicated above.

Note that no support for this new device value was added to any
hypervisor drivers other than qemu, because it's unclear what it might
mean (if anything) to those drivers.

This patch adds two capabilities flags to deal with various aspects
of supporting SG_IO commands on virtio-blk-pci devices:

  QEMU_CAPS_VIRTIO_BLK_SCSI
    set if -device virtio-blk-pci accepts the scsi="on|off" option
    When present, this is on by default, but can be set to off to disable
    SG_IO functions.

  QEMU_CAPS_VIRTIO_BLK_SG_IO
    set if SG_IO commands are supported in the virtio-blk-pci driver
    (present since qemu 0.11 according to a qemu developer, if I
     understood correctly)

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=638633

Although scripts are not used by interfaces of type other than
"ethernet" in qemu, due to the fact that the parser stores the script
name in a union that is only valid when type is ethernet or bridge,
there is no way for anyone except the parser itself to catch the
problem of specifying an interface script for an inappropriate
interface type (by the time the parsed data gets back to the code that
called the parser, all evidence that a script was specified is
forgotten).

Since the parser itself should be agnostic to which type of interface
allows scripts (an example of why: a script specified for an interface
of type bridge is valid for xen domains, but not for qemu domains),
the solution here is to move the script out of the union(s) in the
DomainNetDef, always populate it when specified (regardless of
interface type), and let the driver decide whether or not it is
appropriate.

Currently the qemu, xen, libxml, and uml drivers recognize the script
parameter and do something with it (the uml driver only to report that
it isn't supported). Those drivers have been updated to log a
CONFIG_UNSUPPORTED error when a script is specified for an interface
type that's inappropriate for that particular hypervisor.

(NB: There was earlier discussion of solving this problem by adding a
VALIDATE flag to all libvirt APIs that accept XML, which would cause
the XML to be validated against the RNG files. One statement during
that discussion was that the RNG shouldn't contain hypervisor-specific
things, though, and a proper solution to this problem would require
that (again, because a script for an interface of type "bridge" is
accepted by xen, but not by qemu).

Commit ae523427 missed one pair of functions that could use
the helper routine.

* src/qemu/qemu_driver.c (qemuSetSchedulerParametersFlags)
(qemuGetSchedulerParametersFlags): Simplify.

* configure.ac docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: updated localizations from transifex and regenerated

On rawhide, gcc is new enough to output new DWARF information that
pdwtags has not yet learned, but the resulting 'make check' output
was rather confusing:

$ make -C src check
...
  GEN    virkeepaliveprotocol-structs
die__process_function: DW_TAG_INVALID (0x4109) @ <0x58c> not handled!
WARNING: your pdwtags program is too old
WARNING: skipping the virkeepaliveprotocol-structs test
WARNING: install dwarves-1.3 or newer
...
$ pdwtags --version
v1.9

I've filed the pdwtags deficiency as
https://bugzilla.redhat.com/show_bug.cgi?id=772358

* src/Makefile.am (PDWTAGS): Don't leave -t file behind on version
mismatch.  Soften warning message, since 1.9 is newer than 1.3.
Don't leak stderr from broken version.

Commit db371a21 mistakenly added new functions inside a #ifndef WIN32
guard, even though they are needed on all platforms.

* src/util/command.c (virCommandFDSet): Move outside WIN32
conditional.