Reported by Alex Jia:

==21503== 112 (32 direct, 80 indirect) bytes in 1 blocks are
definitely lost in loss record 37 of 40
==21503==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==21503==    by 0x4A8991: virAlloc (memory.c:101)
==21503==    by 0x505A6C: x86DataCopy (cpu_x86.c:247)
==21503==    by 0x507B34: x86Compute (cpu_x86.c:1225)
==21503==    by 0x43103C: qemuBuildCommandLine (qemu_command.c:3561)
==21503==    by 0x41C9F7: testCompareXMLToArgvHelper
(qemuxml2argvtest.c:183)
==21503==    by 0x41E10D: virtTestRun (testutils.c:141)
==21503==    by 0x41B942: mymain (qemuxml2argvtest.c:705)
==21503==    by 0x41D7E7: virtTestMain (testutils.c:696)

This is probably not strictly needed as save operation is not live but
we may have other reasons to avoid blocking qemu's main loop.

In case the caller specifies that confined guests are required but the
security driver turns out to be 'none', we should return an error since
this driver clearly cannot meet that requirement.  As a result of this
error, libvirtd fails to start when the host admin explicitly sets
confined guests are required but there is no security driver available.

Since security driver 'none' cannot create confined guests, we override
default confined setting so that hypervisor drivers do not thing they
should create confined guests.

Security label type 'none' requires relabel to be set to 'no' so there's
no reason to output this extra attribute.  Moreover, since relabel is
internally stored in a negative from (norelabel), the default value for
relabel would be 'yes' in case there is no <seclabel> element in domain
configuration.  In case VIR_DOMAIN_SECLABEL_DEFAULT turns into
VIR_DOMAIN_SECLABEL_NONE, we would incorrectly output relabel='yes' for
seclabel type 'none'.

Qemu uses non-blocking I/O which doesn't play nice with regular file
descriptors. We need to pass a pipe to qemu instead, which can easily be
done using iohelper.

virFileDirectFd was used for accessing files opened with O_DIRECT using
libvirt_iohelper. We will want to use the helper for accessing files
regardless on O_DIRECT and thus virFileDirectFd was generalized and
renamed to virFileWrapperFd.

Any device XML doesn't use the same order as libvirt generates, or
uses decimal for attributes like "slot" of "<address>" will cause
device detaching to fail, as virsh compares the XML simply earlier
in strict manner before internal parsing.

This is regression introduced by ea7182c2.

Gcc warned about an unused static function.

* python/libvirt-qemu-override.c (py_str): Delete.

 dmidecode displays processor information, followed by BIOS, system and
 memory-DIMM details.
 Calls to virSysinfoParseBIOS(), virSysinfoParseSystem() would update
 the buffer pointer 'base', so the processor information would be lost
 before virSysinfoParseProcessor() was called. Sysinfo would therefore
 not be able to display processor details -- It only described <bios>,
 <system> and <memory_device> details.
 This patch attempts to insulate sysinfo from ordering of dmidecode
 output.

Before the fix:
---------------
virsh # sysinfo
<sysinfo type='smbios'>
  <bios>
    ....
  </bios>
  <system>
    ....
  </system>
  <memory_device>
    ....
  </memory_device>

After the fix:
-------------
virsh # sysinfo
<sysinfo type='smbios'>
  <bios>
    ....
  </bios>
  <system>
    ....
  </system>
  <processor>
    ....
  </processor>
  <memory_device>
    ....
  </memory_device>

Input to the volume cloning code is a source volume and an XML
descriptor for the new volume. It is possible for the new volume
to have a greater size than source volume, at which point libvirt
will just stick 0s on the end of the new image (for raw format
anyways).

Unfortunately a logic error messed up our tracking of the of the
excess amount that needed to be written: end result is that sparse
clones were made very much non-sparse, and cloning regular disk
images could end up excessively sized (though data unaltered).

Drop the 'remain' variable entriely here since it's redundant, and
track actual allocation directly against the desired 'total'.

See: https://bugzilla.redhat.com/show_bug.cgi?id=785269

The specfile requires avahi during install if libvirt was built with
avahi support, but there are many situations where it is undesirable
to install avahi due to security concerns. This patch requires only
the avahi-libs package, which is needed by libvirt to call the
function that tries to attach to the avahi daemon, but will instead
silently fail because the avahi-daemon is in the main avahi package,
and that package isn't installed.

If you are sitting in front of a physical machine and logged in as
a regular user, you can connect to the system libvirtd instance
by providing a root password to policykit. This is how most
virt-manager users talk to libvirt.

However, if you are launching virt-manager over ssh -X, or over
VNC started from say /etc/sysconfig/vncservers, our policykit policy
rejects the user outright, providing no option to provide the root
password. This is confusing to users and doesn't seem to serve much
point.

Change the policy to allow inactive (VNC) and non-local (SSH, VNC)
to provide root credentials for accessing system libvirtd. We use
auth_admin rather than auth_admin_keep so that credentials aren't
cached at all, and every subsequent reconnection to libvirt requires
auth.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=625115
Similar change to PackageKit policy:
https://bugzilla.redhat.com/show_bug.cgi?id=528511

As we already link with libvirt.la which contains libvirt_utils.la.
Double linking causes global symbols to be presented twice and
thus confusion. This partially reverts c700613b

gcc 4.7 complains:

util/virhashcode.c:49:17: error: always_inline function might not be inlinable [-Werror=attributes]
util/virhashcode.c:35:17: error: always_inline function might not be inlinable [-Werror=attributes]

Normal 'inline' is a hint that the compiler may ignore; the fact
that the function is static is good enough.  We don't care if the
compiler decided not to inline after all.

* src/util/virhashcode.c (getblock, fmix): Relax attribute.

Someone mentioned to me that they interpreted this section of the KVM
driver page as suggesting that new guests should be created by
creating a qemu commandline and converting it to XML with
domxml-from-native.  I don't think that's the intent of
domxml-from-native, so I added that clarification.

filename is not initialized to NULL while it's unconditionally freed in
the error path.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

On CentOS5:
If "virsh edit $DOM" is used and an error happens (for example changing
any live cycle action to a non-existing value), libvirt forgets that
$DOM exists, since it is already removed from the internal hash tables,
which are used for domain lookup.
In once case (unreproducible) even the persistent configuration
/etc/xen/$DOM was deleted.

Instead of using the compound function xenXMConfigSaveFile() explicitly
use xenFomatXM() and virConfWriteFile() to distinguish between a failure
in converting the libvirt definition to the xen-xm format and a problem
when writing the file.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Commit b170eb99 introduced a bug: domains that had an explicit
<seclabel type='none'/> when started would not be reparsed if
libvirtd restarted.  It turns out that our testsuite was not
exercising this because it never tried anything but inactive
parsing.  Additionally, the live XML for such a domain failed
to re-validate.  Applying just the tests/ portion of this patch
will expose the bugs that are fixed by the other two files.

* docs/schemas/domaincommon.rng (seclabel): Allow relabel under
type='none'.
* src/conf/domain_conf.c (virSecurityLabelDefParseXML): Per RNG,
presence of <seclabel> with no type implies dynamic.  Don't
require sub-elements for type='none'.
* tests/qemuxml2xmltest.c (mymain): Add test.
* tests/qemuxml2argvtest.c (mymain): Likewise.
* tests/qemuxml2argvdata/qemuxml2argv-seclabel-none.xml: Add file.
* tests/qemuxml2argvdata/qemuxml2argv-seclabel-none.args: Add file.
Reported by Ansis Atteka.

Commit 8f00276c consolidated other
.gitignore files to the master one, but forgot to add some test output
files.

Move myself from 'Previous maintainers' section to 'the primary maintainers and
people with commit access rights' section, because I have a commit rights now.
Signed-off-by: NAlex Jia <ajia@redhat.com>

Commit fad5cd21 introduces a new flag
that allows to show domain's title with domains. This commit introduced
resource leak while listing inactive domains with titles.

On CentOS5 with xen-3.0.3:

 Program received signal SIGSEGV, Segmentation fault.
 virFree (ptrptr=0x8) at util/memory.c:310
 310         free(*(void**)ptrptr);
 (gdb) bt
 #0  virFree (ptrptr=0x8) at util/memory.c:310
 #1  0x00002aaaaae167c8 in xenXMDomainDefineXML (conn=0x694e80, xml=0x6b2ce0 "P\fk") at xen/xm_internal.c:1199
 #2  0x00002aaaaae070d7 in xenUnifiedDomainDefineXML (conn=0x8,
     xml=0x6ac040 "<domain type='xen'>\n  <name>pv</name>\n  <uuid>20291bc0-453a-4d6c-c6ac-4e5af63b932c</uuid>\n  <memory>1048576</memory>\n  <currentMemory>1048576</currentMemory>\n  <vcpu>1</vcpu>\n  <os>\n    <type arch='x8"...) at xen/xen_driver.c:1524
 #3  0x00002aaaaada7803 in virDomainDefineXML (conn=0x694e80,
     xml=0x6ac040 "<domain type='xen'>\n  <name>pv</name>\n  <uuid>20291bc0-453a-4d6c-c6ac-4e5af63b932c</uuid>\n  <memory>1048576</memory>\n  <currentMemory>1048576</currentMemory>\n  <vcpu>1</vcpu>\n  <os>\n    <type arch='x8"...) at libvirt.c:7823
 #4  0x0000000000426173 in cmdEdit (ctl=0x7fffffffb8e0, cmd=<value optimized out>) at virsh.c:14882
 #5  0x000000000041c9ce in vshCommandRun (ctl=0x7fffffffb8e0, cmd=0x658c50) at virsh.c:17712
 #6  0x000000000042c3b9 in main (argc=1, argv=<value optimized out>) at virsh.c:19317
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Also encourages people to use per-device boot elements for better
control.

Calling qemuDomainMigrateGraphicsRelocate notifies spice clients to
connect to destination qemu so that they can seamlessly switch streams
once migration is done. Unfortunately, current qemu is not able to
accept any connections while incoming migration connection is open.
Thus, we need to delay opening the migration connection to the point
spice client is already connected to the destination qemu.

Typo introduced by c18a88ac

Detected by valgrind. the codes are allocating 0 bytes memory to variable
cpumap by vshCalloc function, and then the function VIR_USE_CPU will access
it later, a invalid read error will be hit.

* tools/virsh.c(cmdVcpuPin): fix invalid read error.

* How to reproduce?
% valgrind -v --read-var-info=yes virsh vcpupin <domain> 0 0

* Actual result:

==27271== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 8 from 6)
==27271==
==27271== 1 errors in context 1 of 2:
==27271== Invalid read of size 1
==27271==    at 0x39CF087E2E: __GI_memcpy (in /lib64/libc-2.12.so)
==27271==    by 0x39CF114FDC: xdrmem_putbytes (in /lib64/libc-2.12.so)
==27271==    by 0x39CF114707: xdr_opaque (in /lib64/libc-2.12.so)
==27271==    by 0x4D56194: xdr_remote_domain_pin_vcpu_args (remote_protocol.c:1844)
==27271==    by 0x4D6CCE1: virNetMessageEncodePayload (virnetmessage.c:341)
==27271==    by 0x4D5A44B: virNetClientProgramCall (virnetclientprogram.c:327)
==27271==    by 0x4D36EDB: callWithFD (remote_driver.c:4546)
==27271==    by 0x4D36F7B: call (remote_driver.c:4567)
==27271==    by 0x4D3B2C1: remoteDomainPinVcpu (remote_client_bodies.h:1566)
==27271==    by 0x4D199D3: virDomainPinVcpu (libvirt.c:8585)
==27271==    by 0x4241F4: cmdVcpuPin (virsh.c:5262)
==27271==    by 0x4150A6: vshCommandRun (virsh.c:17712)
==27271==  Address 0x5602b80 is 0 bytes after a block of size 0 alloc'd
==27271==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==27271==    by 0x4C89BDF: virAllocN (memory.c:129)
==27271==    by 0x423868: _vshCalloc.clone.2 (virsh.c:454)
==27271==    by 0x423EF9: cmdVcpuPin (virsh.c:5190)
==27271==    by 0x4150A6: vshCommandRun (virsh.c:17712)
==27271==    by 0x426583: main (virsh.c:19289)
==27271==
==27271==
==27271== 4 errors in context 2 of 2:
==27271== Invalid read of size 1
==27271==    at 0x424133: cmdVcpuPin (virsh.c:5245)
==27271==    by 0x4150A6: vshCommandRun (virsh.c:17712)
==27271==    by 0x426583: main (virsh.c:19289)
==27271==  Address 0x5602b80 is 0 bytes after a block of size 0 alloc'd
==27271==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==27271==    by 0x4C89BDF: virAllocN (memory.c:129)
==27271==    by 0x423868: _vshCalloc.clone.2 (virsh.c:454)
==27271==    by 0x423EF9: cmdVcpuPin (virsh.c:5190)
==27271==    by 0x4150A6: vshCommandRun (virsh.c:17712)
==27271==    by 0x426583: main (virsh.c:19289)
Signed-off-by: NAlex Jia <ajia@redhat.com>

Unlike .cvsignore under CVS, git allows for ignoring nested
names.  We weren't very consistent where new tests were
being ignored (some in .gitignore, some in tests/.gitignore),
and I found it easier to just consolidate everything.

* .gitignore: Subsume entries from subdirectories.
* daemon/.gitignore: Delete.
* docs/.gitignore: Likewise.
* docs/devhelp/.gitignore: Likewise.
* docs/html/.gitignore: Likewise.
* examples/dominfo/.gitignore: Likewise.
* examples/domsuspend/.gitignore: Likewise.
* examples/hellolibvirt/.gitignore: Likewise.
* examples/openauth/.gitignore: Likewise.
* examples/domain-events/events-c/.gitignore: Likewise.
* include/libvirt/.gitignore: Likewise.
* src/.gitignore: Likewise.
* src/esx/.gitignore: Likewise.
* tests/.gitignore: Likewise.
* tools/.gitignore: Likewise.

This eliminates the warning message reported in:

 https://bugzilla.redhat.com/show_bug.cgi?id=624447

It was caused by a failure to open an image file that is not
accessible by root (the uid libvirtd is running as) because it's on a
root-squash NFS share, owned by a different user, with permissions of
660 (or maybe 600).

The solution is to use virFileOpenAs() rather than open(). The
codepath that generates the error is during qemuSetupDiskCGroup(), but
the actual open() is in a lower-level generic function called from
many places (virDomainDiskDefForeachPath), so some other pieces of the
code were touched just to add dummy (or possibly useful) uid and gid
arguments.

Eliminating this warning message has the nice side effect that the
requested operation may even succeed (which in this case isn't
necessary, but shouldn't hurt anything either).

virFileOpenAs previously would only try opening a file as the current
user, or as a different user, but wouldn't try both methods in a
single call. This made it cumbersome to use as a replacement for
open(2). Additionally, it had a lot of historical baggage that led to
it being difficult to understand.

This patch refactors virFileOpenAs in the following ways:

* reorganize the code so that everything dealing with both the parent
  and child sides of the "fork+setuid+setgid+open" method are in a
  separate function. This makes the public function easier to understand.

* Allow a single call to virFileOpenAs() to first attempt the open as
  the current user, and if that fails to automatically re-try after
  doing fork+setuid (if deemed appropriate, i.e. errno indicates it
  would now be successful, and the file is on a networkFS). This makes
  it possible (in many, but possibly not all, cases) to drop-in
  virFileOpenAs() as a replacement for open(2).

  (NB: currently qemuOpenFile() calls virFileOpenAs() twice, once
  without forking, then again with forking. That unfortunately can't
  be changed without at least some discussion of the ramifications,
  because the requested file permissions are different in each case,
  which is something that a single call to virFileOpenAs() can't deal
  with.)

* Add a flag so that any fchown() of the file to a different uid:gid
  is explicitly requested when the function is called, rather than it
  being implied by the presence of the O_CREAT flag. This just makes
  for less subtle surprises to consumers. (Commit
  b1643dc1 added the check for O_CREAT
  before forcing ownership. This patch just makes that restriction
  more explicit.)

* If either the uid or gid is specified as "-1", virFileOpenAs will
  interpret this to mean "the current [gu]id".

All current consumers of virFileOpenAs should retain their present
behavior (after a few minor changes to their setup code and
arguments).

Rename the src/util/netlink files to src/util/virnetlink to
better fit the naming scheme. Also rename nlComm to virNetlinkCommand.
Signed-off-by: ND. Herrendoerfer <d.herrendoerfer@herrendoerfer.name>

This allows virsh to use the new VIR_DOMAIN_DESTROY_GRACEUL flag for
virDomainDestroyFlags.

When libvirt's virDomainDestroy API is shutting down the qemu process,
it first sends SIGTERM, then waits for 1.6 seconds and, if it sees the
process still there, sends a SIGKILL.

There have been reports that this behavior can lead to data loss
because the guest running in qemu doesn't have time to flush its disk
cache buffers before it's unceremoniously whacked.

This patch maintains that default behavior, but provides a new flag
VIR_DOMAIN_DESTROY_GRACEFUL to alter the behavior. If this flag is set
in the call to virDomainDestroyFlags, SIGKILL will never be sent to
the qemu process; instead, if the timeout is reached and the qemu
process still exists, virDomainDestroy will return an error.

Once this patch is in, the recommended method for applications to call
virDomainDestroyFlags will be with VIR_DOMAIN_DESTROY_GRACEFUL
included. If that fails, then the application can decide if and when
to call virDomainDestroyFlags again without
VIR_DOMAIN_DESTROY_GRACEFUL (to force the issue with SIGKILL).

(Note that this does not address the issue of existing applications
that have not yet been modified to use VIR_DOMAIN_DESTROY_GRACEFUL.
That is a separate patch.)

compat{a->i}bility
erron{->e}ous
nec{c->}essary.
Either "the" or "a".
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Two types of memory stats were not reported by python bindings. This
patch fixes both of them.

This patch starts the process of elevating the python binding code
to be on the same level as the rest of libvirt when it comes to
requiring good coding styles.  Statically linking against the
libvirt_util library makes it much easier to write good code,
rather than having to open-code and reinvent things locally.

Done by global search and replace of s/free(/VIR_FREE(/, followed
by hand-inspection of remaining malloc and redundant memset.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_raw_allocation):
Remove python from exemption.
* python/Makefile.am (INCLUDES): Add gnulib and src/util.  Drop
$(top_builddir)/$(subdir), as automake already guarantees that.
(mylibs, myqemulibs): Pull in libvirt_util and gnulib.
(libvirtmod_la_CFLAGS): Catch compiler warnings if configured to
use -Werror.
* python/typewrappers.c (libvirt_charPtrSizeWrap)
(libvirt_charPtrWrap): Convert free to VIR_FREE.
* python/generator.py (print_function_wrapper): Likewise.
* python/libvirt-override.c: Likewise.

The bulk of this patch was done with:

sed -i 's/\(\bfree *(/VIR_FREE(/g' tests/*.c

followed by fixing the few compile errors that resulted.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_raw_allocation):
Remove tests from exemption.
* tests/testutils.h: Add common header.
* tests/commandhelper.c: Fix offenders.
* tests/cputest.c: Likewise.
* tests/domainsnapshotxml2xmltest.c: Likewise.
* tests/interfacexml2xmltest.c: Likewise.
* tests/networkxml2argvtest.c: Likewise.
* tests/networkxml2xmltest.c: Likewise.
* tests/nodedevxml2xmltest.c: Likewise.
* tests/nodeinfotest.c: Likewise.
* tests/nwfilterxml2xmltest.c: Likewise.
* tests/qemuargv2xmltest.c: Likewise.
* tests/qemuxml2argvtest.c: Likewise.
* tests/qemuxml2xmltest.c: Likewise.
* tests/qemuxmlnstest.c: Likewise.
* tests/qparamtest.c: Likewise.
* tests/sexpr2xmltest.c: Likewise.
* tests/storagepoolxml2xmltest.c: Likewise.
* tests/storagevolxml2xmltest.c: Likewise.
* tests/testutils.c: Likewise.
* tests/virshtest.c: Likewise.
* tests/xencapstest.c: Likewise.
* tests/xmconfigtest.c: Likewise.
* tests/xml2sexprtest.c: Likewise.

Our HACKING discourages use of malloc and free, for at least
a couple of years now.  But we weren't enforcing it, until now :)

For now, I've exempted python and tests, and will clean those up
in subsequent patches.  Examples should be permanently exempt,
since anyone copying our examples won't have use of our
internal-only memory.h via libvirt_util.la.

* cfg.mk (sc_prohibit_raw_allocation): New rule.
(exclude_file_name_regexp--sc_prohibit_raw_allocation): and
exemptions.
* src/cpu/cpu.c (cpuDataFree): Avoid false positive.
* src/conf/network_conf.c (virNetworkDNSSrvDefParseXML): Fix
offenders.
* src/libxl/libxl_conf.c (libxlMakeDomBuildInfo, libxlMakeVfb)
(libxlMakeDeviceModelInfo): Likewise.
* src/rpc/virnetmessage.c (virNetMessageSaveError): Likewise.
* tools/virsh.c (_vshMalloc, _vshCalloc): Likewise.

I noticed some redundant code while preparing my next patch.

* python/generator.py (py_types): Fix 'const char *' mapping.
* python/typewrappers.h (libvirt_charPtrConstWrap): Drop.
* python/typewrappers.c (libvirt_charPtrConstWrap): Delete, since
it is identical to libvirt_constcharPtrWrap.