We publish libvirt-api.xml for others to use, and in fact, the
libvirt-python bindings use it to generate python constants that
correspond to our enum values.  However, we had an off-by-one bug
that any enum that relied on C's rules for implicit initialization
of the first enum member to 0 got listed in the xml as having a
value of 1 (and all later members of the enum were equally
botched).

The fix is simple - since we add one to the previous value when
encountering an enum without an initializer, the previous value
must start at -1 so that the first enum member is assigned 0.

The python generator code has had the off-by-one ever since DV
first wrote it years ago, but most of our public enums were immune
because they had an explicit = 0 initializer.  The only affected
enums are:
- virDomainEventGraphicsAddressType (such as
VIR_DOMAIN_EVENT_GRAPHICS_ADDRESS_IPV4), since commit 987e31ed
(libvirt v0.8.0)
- virDomainCoreDumpFormat (such as VIR_DOMAIN_CORE_DUMP_FORMAT_RAW),
since commit 9fbaff00 (libvirt v1.2.3)
- virIPAddrType (such as VIR_IP_ADDR_TYPE_IPV4), since commit
03e0e79e (not yet released)

Thanks to Nehal J Wani for reporting the problem on IRC, and
for helping me zero in on the culprit function.

* docs/apibuild.py (CParser.parseEnumBlock): Fix implicit enum
values.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 9b291bbe)

When creating a new disk mirror the new struct is stored in a separate
variable until everything went well. The removed hunk would actually
remove existing mirror information for example when the api would be run
if a mirror still exists.

(cherry picked from commit 02b364e1)

This fixes a regression introduced in commit ff5f30b6.
Signed-off-by: NEric Blake <eblake@redhat.com>

Conflicts:
	src/qemu/qemu_driver.c - no refactoring of commit 7b7bf001

Jim Fehlig reported a regression found by libvirt-TCK tests:

> ~ # perl /usr/share/libvirt-tck/tests/qemu/100-disk-encryption.t
...
> ok 4 - defined persistent domain config
> # Starting inactive domain config
> libvirt error code: 1, message: internal error: unable to execute QEMU command
> 'cont': 'drive-ide0-0-1'
> (/var/cache/libvirt-tck/300-disk-encryption/demo.qcow2) is encrypted

Commit 2279d560 converted a boolean into a pointer with the intent of
transferring that pointer out of a temporary object into the caller's
data structure.  The temporary structure meant that meta->encryption
was always NULL on entry, so we could get away with blindly allocating
the pointer when the header said so.  But later, commit 8823272d
tweaked things to do backing chain detection in-place, rather than via
a temporary object; this has the net result that meta->encryption can
be non-NULL on entry.  Not only did this turn the latent behavior into
a memory leak, it is also a behavior regression: blindly allocating a
new pointer wipes out what secrets we already knew about the chain,
making it impossible to restart the domain.

Of course, no one in their right mind should be relying on qcow2
encryption - it is fundamentally flawed.  And sadly, the TCK tests
don't get run often enough, and this shows that our virstoragetest
does not exercise encrypted images at all.  Otherwise, we could
have avoided a release containing this regression.

* src/util/virstoragefile.c (virStorageFileGetMetadataInternal):
Don't nuke an already-existing encryption.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 1c7eb95c)

If the XML_PARSE_NOENT flag is passed to libxml2, then any
entities in the input document will be fully expanded. This
allows the user to read arbitrary files on the host machine
by creating an entity pointing to a local file. Removing
the XML_PARSE_NOENT flag means that any entities are left
unchanged by the parser, or expanded to "" by the XPath
APIs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit d6b27d3e)

* docs/news.html.in libvirt.spec.in: updates for release
* po/*.po*: fetched new localization and regenerated

This fixes link failures like:

  CCLD     virfirewalltest
  /usr/bin/ld: virfirewalltest-virfirewalltest.o: undefined reference to
  symbol 'dbus_message_iter_init_append'

When building packages in a clean chroot the QEMU_USER and QEMU_GROUP
don't exist making VirQemuDriverConfigNew fail with privileged=true.

Avoid that by not requiring privileged mode upfront but setting it later
so we skip the user/group existence check.

This solution was suggested by Daniel P. Berrange and tested by Martin
Kletzander.

Currently firewalling is supported on Linux only, so skip the
virfirewalltest on other platforms.

Commit id 'ac9a0963' refactored out the 'withCapacity' for the
virStorageBackendUpdateVolInfo() API.  See:

http://www.redhat.com/archives/libvir-list/2014-April/msg00043.html

This resulted in a difference in how 'virsh vol-info --pool <poolName>
<volume>' or 'virsh vol-list vol-list --pool <poolName> --details' outputs
the capacity information for a directory pool with a qcow2 sparse file.

For example, using the following XML

mkdir /home/TestPool
cat testpool.xml
<pool type='dir'>
  <name>TestPool</name>
  <uuid>6bf80895-10b6-75a6-6059-89fdea2aefb7</uuid>
  <source>
  </source>
  <target>
    <path>/home/TestPool</path>
    <permissions>
      <mode>0755</mode>
      <owner>0</owner>
      <group>0</group>
    </permissions>
  </target>
</pool>

virsh pool-create testpool.xml
virsh vol-create-as --pool TestPool temp_vol_1 \
      --capacity 1048576 --allocation 1048576 --format qcow2
virsh vol-info --pool TestPool temp_vol_1

Results in listing a Capacity value.  Prior to the commit, the value would
be '1.0 MiB' (1048576 bytes). However, after the commit the output would be
(for example) '192.50 KiB', which for my system was the size of the volume
in my file system (eg 'ls -l TestPool/temp_vol_1' results in '197120' bytes
or 192.50 KiB). While perhaps technically correct, it's not necessarily
what the user expected (certainly virt-test didn't expect it).

This patch restores the code to not update the target capacity for this path

gnutls-3.3.0 and newer leaves 2 FDs open in order to be backwards
compatible when it comes to chrooted binaries [1].  Linking
commandhelper with gnutls then leaves these two FDs open and
commandtest fails thanks to that.  This patch does not link
commandhelper with libvirt.la, but rather only the utilities making
the test pass.

Based on suggestion from Daniel [2].

[1] http://lists.gnutls.org/pipermail/gnutls-help/2014-April/003429.html
[2] https://www.redhat.com/archives/libvir-list/2014-April/msg01119.htmlSigned-off-by: NMartin Kletzander <mkletzan@redhat.com>

Older gcc (4.1.2-55.el5, 4.2.1 on FreeBSD) reports bogus warnings:
../../src/conf/nwfilter_conf.c:2111: warning: 'protocol' may be used
uninitialized in this function
../../src/conf/nwfilter_conf.c:2110: warning: 'dataProtocolID' may be
used uninitialized in this function

Initialize them to NULL to make the compiler happy.

Commit f22b7899 stumbled across a difference between 32-bit and
64-bit platforms when parsing "-1" as an int.  Now that we've
fixed that difference, it's time to fix the testsuite.

* src/util/virstoragefile.c (virStorageFileParseChainIndex):
Require a positive index.
Signed-off-by: NEric Blake <eblake@redhat.com>

strtoul() is required to parse negative numbers as their
twos-complement positive counterpart.  But sometimes we want
to reject negative numbers.  Add new functions to do this.
The 'p' suffix is a mnemonic for 'positive' (technically it
also parses 0, but 'non-negative' doesn't lend itself to a
nice one-letter suffix).

* src/util/virstring.h (virStrToLong_uip, virStrToLong_ulp)
(virStrToLong_ullp): New prototypes.
* src/util/virstring.c (virStrToLong_uip, virStrToLong_ulp)
(virStrToLong_ullp): New functions.
* src/libvirt_private.syms (virstring.h): Export them.
* tests/virstringtest.c (testStringToLong): Test them.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit f22b7899 called to light a long-standing latent bug: the
behavior of virStrToLong_ui was different on 32-bit platforms
than on 64-bit platforms.  Curse you, C type promotion and
narrowing rules, and strtoul specification.  POSIX says that for
a 32-bit long, strtol handles only 2^32 values [LONG_MIN to
LONG_MAX] while strtoul handles 2^33 - 1 values [-ULONG_MAX to
ULONG_MAX] with twos-complement wraparound for negatives.  Thus,
parsing -1 as unsigned long produces ULONG_MAX, rather than a
range error.  We WANT[1] this same shortcut for turning -1 into
UINT_MAX when parsing to int; and get it for free with 32-bit
long.  But with 64-bit long, ULONG_MAX is outside the range
of int and we were rejecting it as invalid; meanwhile, we were
silently treating -18446744073709551615 as 1 even though it
textually exceeds INT_MIN.  Too bad there's not a strtoui() in
libc that does guaranteed parsing to int, regardless of the size
of long.

The bug has been latent since 2007, introduced by Jim Meyering
in commit 5d254191 in the attempt to eradicate unsafe use of
strto[u]l when parsing ints and longs.  How embarrassing that we
are only discovering it now - so I'm adding a testsuite to ensure
that it covers all the corner cases we care about.

[1] Ideally, we really want the caller to be able to choose whether
to allow negative numbers to wrap around to their 2s-complement
counterpart, as in strtoul, or to force a stricter input range
of [0 to UINT_MAX] by rejecting negative signs; this will be added
in a later patch for all three int types.

This patch is tested on both 32- and 64-bit; the enhanced
virstringtest passes on both platforms, while virstoragetest now
reliably fails on both platforms instead of just 32-bit platforms.
That test will be fixed later.

* src/util/virstring.c (virStrToLong_ui): Ensure same behavior
regardless of platform long size.
* tests/virstringtest.c (testStringToLong): New function.
(mymain): Comprehensively test string to long parsing.
Signed-off-by: NEric Blake <eblake@redhat.com>

A couple of places in the QEMU XML -> ARGV conversion code
raised an error but then forgot to return an error status
due to missing gotos. While fixing this also tweak style
of a couple of other error reports
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If a domain network interface that contains a <filterref> is modified
"live" using "virsh update-device --live", libvirtd would crash. This
was because the code supporting live update of an interface's
filterref was assuming that a filterref might be added or modified,
but didn't account for removing the filterref, resulting in a null
dereference of the filter name.

Introduced with commit 258fb278, which was first in libvirt v1.0.1.

This addresses https://bugzilla.redhat.com/show_bug.cgi?id=1093301

To avoid memory leak of the "backingStoreRaw" field when reparsing
backing chains a new function is being introduced by this patch that
shall be used to clear backing store information.

The memory leak was introduced in commit 8823272d.

Refactor the ebiptablesTearNewRules function so that the teardown of temporary
filters can also be called by the ebiptablesAllTeardown function.

This fixes a problem that leaves temporary filters behind when a VM shuts down
while its filters are modified.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>

v1->v2:
  - test cases adjusted to expect more commands

https://bugzilla.redhat.com/show_bug.cgi?id=808463

Well, libvirt doesn't distinguish between domain poweroff and
hibernation (S4). It's hard to differentiate these two on a real
machine anyway. As a result, any device that is hot(un-)plugged is
lost (appears again) when domain is started again as from our POV
it is a fresh cold boot. Instead of doing anything wise here, we
should just document this as known limitation.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The LXC controller itself needs to mknod the USB device
node in /dev/bus/usb, so we can't block mknod permission
from the cgroup.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

An IP or IPv6 rule with port specification but without protocol
specification cannot be instantiated by ebtables. The documentation
points to 'protocol' being required but implementation does not
enforce it to be given.

Implement a rule validation function that checks whether the rule is
valid when it is defined. This for example prevents the definition
of rules like:

<ip dstportstart='53'>

where a protocol attribute would be required for it to be valid and for
ebtables to be able to instantiate it. A valid rule then is:

<ip protocol='udp' dstportstart='53'>
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>

This commit adds a new example to illustrate peer to
peer domain migration with virDomainMigrateToURI.
Signed-off-by: NSahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>

This commit provides the ability to virDomainMigrateToURI to
check for SASL credentials when attempts to migrate a domain
with the driver QEMU.
Signed-off-by: NSahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>

We need to include the testutils.h also for freebsd.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Freebsd doesn't know ENODATA so we have to use different EINVAL
error code.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Commit 5c43e2e0 introduced a NULL deref if there is a failure
in virStorageFileGetMetadataInternal.

* src/util/virstoragefile.c (virStorageFileGetMetadataFromBuf):
Fix error handling.
Signed-off-by: NEric Blake <eblake@redhat.com>

We don't support building libvirtd on Win32 since we lack the
fork/exec feature needed for the stateful drivers. Disable this
by default, so users can just do 'mingw32-configure' with no
special args required.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

SO_REUSEADDR on Windows is actually akin to SO_REUSEPORT
on Linux/BSD. ie it allows 2 apps to listen to the same
port at once. Thus we must not set it on Win32 platforms

See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740621.aspxSigned-off-by: NDaniel P. Berrange <berrange@redhat.com>

When EIO comes to qemu while it's replying to
qemuMigrationUpdateJobStatus(), qemu blocks, the migration of RAM can
complete in the meantime, and when qemu unblocks, it sends us
BLOCK_IO_ERROR plus migrations "status": "complete".  Even though we
act upon the BLOCK_IO_ERROR by setting the proper state of the domain,
the call still waits for the proper reply on monitor for query_migrate
and after it gets it, it checks that migration is completed and the
migration is finished.  This is what abort_on_error flag was meant for
(we can migrate with these errors, but this flag must inhibit such
behaviour).  Changing the order of the steps guarantees the flag works
properly.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1045833Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Based on suggestion from Eric [1], because it might not get cleaned up
before the release, so to avoid potential errors.

[1] https://www.redhat.com/archives/libvir-list/2014-April/msg00929.htmlSigned-off-by: NMartin Kletzander <mkletzan@redhat.com>

commit ddf2dfa1
provided a way to determine which bios files to use.
But we need to update related docs.

disccussed at:
http://www.redhat.com/archives/libvir-list/2012-June/msg01286.htmlSigned-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The stripe_unit and stripe_count arguments are passed to rbd_create3 in
the wrong order, resulting in a stripe size of 1 byte with 4194304
stripes on newly created RBD volumes.

https://bugzilla.redhat.com/show_bug.cgi?id=1092208Signed-off-by: NSteven McDonald <steven.mcdonald@anchor.net.au>

Now that all clients have been adjusted, ensure that no future
misuse of readdir is introduced into the code base.

* cfg.mk (sc_prohibit_readdir): New rule.
* src/util/virfile.c (virDirRead): Exempt the wrapper.
Signed-off-by: NEric Blake <eblake@redhat.com>

This attempts to follow the same variable name and usage patterns as
the other conversions to virDirRead().

More instances of failure to report (unlikely) readdir errors.
In one case, I chose to ignore them, given that a readdir error
would be no different than timing out on the loop, where the
fallback path behaves correctly either way.

* src/storage/storage_backend.c (virStorageBackendStablePath):
Ignore readdir errors.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemRefresh): Report readdir errors.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIGetHostNumber): Likewise.
* src/storage/storage_backend_scsi.c (getNewStyleBlockDevice)
(getBlockDevice, virStorageBackendSCSIFindLUs): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Convert all remaining clients of readdir to use the new
interface, so that we can ensure (unlikely) errors while
reading a directory are reported.

* src/openvz/openvz_conf.c (openvzAssignUUIDs): Use new
interface.
* src/parallels/parallels_storage.c (parallelsFindVolumes)
(parallelsFindVmVolumes): Report readdir failures.
* src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Ignore readdir
failures.
* src/secret/secret_driver.c (loadSecrets): Likewise.
* src/qemu/qemu_hostdev.c
(qemuHostdevHostSupportsPassthroughVFIO): Report readdir failures.
* src/xen/xen_inotify.c (xenInotifyOpen): Likewise.
* src/xen/xm_internal.c (xenXMConfigCacheRefresh): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

In making the conversion to the new API, I fixed a couple bugs:
virSCSIDeviceGetSgName would leak memory if a directory
unexpectedly contained multiple entries;
virNetDevTapGetRealDeviceName could report a spurious error
from a stale errno inherited before starting the readdir search.

The decision on whether to store the result of virDirRead into
a variable is based on whether the end of the loop falls through
to cleanup code automatically.  In some cases, we have loops that
are documented to return NULL on failure, and which raise an
error on most failure paths but not in the case where the directory
was unexpectedly empty; it may be worth a followup patch to
explicitly report an error if readdir was successful but the
directory was empty, so that a NULL return always has an error set.

* src/util/vircgroup.c (virCgroupRemoveRecursively): Use new
interface.
(virCgroupKillRecursiveInternal, virCgroupSetOwner): Report
readdir failures.
* src/util/virfile.c (virFileLoopDeviceOpenSearch)
(virFileNBDDeviceFindUnused, virFileDeleteTree): Use new
interface.
* src/util/virnetdevtap.c (virNetDevTapGetRealDeviceName):
Properly check readdir errors.
* src/util/virpci.c (virPCIDeviceIterDevices)
(virPCIDeviceFileIterate, virPCIGetNetName): Report readdir
failures.
(virPCIDeviceAddressIOMMUGroupIterate): Use new interface.
* src/util/virscsi.c (virSCSIDeviceGetSgName): Report readdir
failures, and avoid memory leak.
(virSCSIDeviceGetDevName): Report readdir failures.
* src/util/virusb.c (virUSBDeviceSearch): Report readdir
failures.
* src/util/virutil.c (virGetFCHostNameByWWN)
(virFindFCHostCapableVport): Report readdir failures.
Signed-off-by: NEric Blake <eblake@redhat.com>

When reading configuration files, we were silently ignoring
directory read failures.  While unlikely, we might as well
report them.

* src/conf/domain_conf.c (virDomainObjListLoadAllConfigs): Report
readdir errors.
* src/conf/network_conf.c (virNetworkLoadAllState)
(virNetworkLoadAllConfigs): Likewise.
* src/conf/nwfilter_conf.c (virNWFilterLoadAllConfigs): Likewise.
* src/conf/storage_conf.c (virStoragePoolLoadAllConfigs):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

This makes sure that errno is reset before readdir is called, even if
the loop does a 'continue'.

This fixes issue with musl libc which sets errno on sscanf. The
following 'continue' makes the errno be set before calling readdir.
Signed-off-by: NNatanael Copa <ncopa@alpinelinux.org>
Signed-off-by: NEric Blake <eblake@redhat.com>

Introduce a wrapper for readdir. This helps us make sure that we always
set errno before calling readdir and it will make sure errors are
properly logged.
Signed-off-by: NNatanael Copa <ncopa@alpinelinux.org>
Signed-off-by: NEric Blake <eblake@redhat.com>