Currently there is a restriction that multi-threaded applications
must manually call virInitialize, before threads start using
libvirt, because it is not thread-safe. By switching it to use
a virOnceControl initializer we gain thread safety, and thus
applications no longer need to manually call it. They can rely
on virConnectOpen invoking it for them.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Win32 platforms don't have SIGKILL defined, but they do have
SIGABRT. Since our virProcess wrapper treats anything which
isn't SIGTERM/SIGINT as equivalent to SIGKILL, just use
SIGABRT on Win32.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add two new APIs virNetServerNewPostExecRestart and
virNetServerPreExecRestart which allow a virNetServerPtr
object to be created from a JSON object and saved to a
JSON object, for the purpose of re-exec'ing a process.

This includes serialization of all registered services
and clients
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add two new APIs virNetServerClientNewPostExecRestart and
virNetServerClientPreExecRestart which allow a virNetServerClientPtr
object to be created from a JSON object and saved to a
JSON object, for the purpose of re-exec'ing a process.

This includes serialization of the connected socket associated
with the client
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add two new APIs virNetServerServiceNewPostExecRestart and
virNetServerServicePreExecRestart which allow a virNetServerServicePtr
object to be created from a JSON object and saved to a
JSON object, for the purpose of re-exec'ing a process.

This includes serialization of the listening sockets associated
with the service
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add two new APIs virNetSocketNewPostExecRestart and
virNetSocketPreExecRestart which allow a virNetSocketPtr
object to be created from a JSON object and saved to a
JSON object, for the purpose of re-exec'ing a process.

As well as saving the state in JSON format, the second
method will disable the O_CLOEXEC flag so that the open
file descriptors are preserved across the process re-exec()

Since it is not possible to serialize SASL or TLS encryption
state, an error will be raised if attempting to perform
serialization on non-raw sockets
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add two new APIs virLockSpaceNewPostExecRestart and
virLockSpacePreExecRestart which allow a virLockSpacePtr
object to be created from a JSON object and saved to a
JSON object, for the purposes of re-exec'ing a process.

As well as saving the state in JSON format, the second
method will disable the O_CLOEXEC flag so that the open
file descriptors are preserved across the process re-exec()
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The previously introduced virFile{Lock,Unlock} APIs provide a
way to acquire/release fcntl() locks on individual files. For
unknown reason though, the POSIX spec says that fcntl() locks
are released when *any* file handle referring to the same path
is closed. In the following sequence

  threadA: fd1 = open("foo")
  threadB: fd2 = open("foo")
  threadA: virFileLock(fd1)
  threadB: virFileLock(fd2)
  threadB: close(fd2)

you'd expect threadA to come out holding a lock on 'foo', and
indeed it does hold a lock for a very short time. Unfortunately
when threadB does close(fd2) this releases the lock associated
with fd1. For the current libvirt use case for virFileLock -
pidfiles - this doesn't matter since the lock is acquired
at startup while single threaded an never released until
exit.

To provide a more generally useful API though, it is necessary
to introduce a slightly higher level abstraction, which is to
be referred to as a "lockspace".  This is to be provided by
a virLockSpacePtr object in src/util/virlockspace.{c,h}. The
core idea is that the lockspace keeps track of what files are
already open+locked. This means that when a 2nd thread comes
along and tries to acquire a lock, it doesn't end up opening
and closing a new FD. The lockspace just checks the current
list of held locks and immediately returns VIR_ERR_RESOURCE_BUSY.

NB, the API as it stands is designed on the basis that the
files being locked are not being otherwise opened and used
by the application code. One approach to using this API is to
acquire locks based on a hash of the filepath.

eg to lock /var/lib/libvirt/images/foo.img the application
might do

   virLockSpacePtr lockspace = virLockSpaceNew("/var/lib/libvirt/imagelocks");
   lockname = md5sum("/var/lib/libvirt/images/foo.img");
   virLockSpaceAcquireLock(lockspace, lockname);

NB, in this example, the caller should ensure that the path
is canonicalized before calculating the checksum.

It is also possible to do locks directly on resources by
using a NULL lockspace directory and then using the file
path as the lock name eg

   virLockSpacePtr lockspace = virLockSpaceNew(NULL);
   virLockSpaceAcquireLock(lockspace, "/var/lib/libvirt/images/foo.img");

This is only safe to do though if no other part of the process
will be opening the files. This will be the case when this
code is used inside the soon-to-be-reposted virlockd daemon
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There was a missing method in python implementation of domain-events
test and this patch adds that.

Given Daniel's announcement[1], code targetting the next release will
be in 1.0.0, not 0.10.3.  Changed mechanically with:

for f in $(git grep -l '0\(.\)10\13\b') ; do
   sed -i -e 's/0\(.\)10\13/1\10\10/g' $f
done

[1]https://www.redhat.com/archives/libvir-list/2012-October/msg00403.html

* docs/formatdomain.html.in: Use 1.0.0 for next release.
* src/interface/interface_backend_udev.c: Likewise.

As approved here:
https://www.redhat.com/archives/libvir-list/2012-October/msg00701.html

* daemon/libvirtd.policy.in: Use LGPLv2+ license.

Add test for 280b8c9e.

There was a crash possible when both <boot dev... and <boot
order... were specified due to virDomainDefParseBootXML() erroring out
before setting *tmp (which was free'd in cleanup).  As a fix, I
created this cleanup that uses one pointer for all the temporary
stored XPath strings and values, plus this pointer is correctly
initialized to NULL.

In commit 9674f2c6, I forgot to change
selabel_lookup with the other functions, so this one-liner does exactly
that.

Detected with:
git grep ';;$' -- '**/*.[ch]'

* src/network/bridge_driver.c (networkRadvdConfContents): Fix
harmless typo.
* src/phyp/phyp_driver.c (phypUUIDTable_Pull): Likewise.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONDriveDel):
Likewise.

BZ:https://bugzilla.redhat.com/show_bug.cgi?id=851981
When using macvtap, a character device gets first created by
kernel with name /dev/tapN, its selinux context is:
system_u:object_r:device_t:s0

Shortly, when udev gets notification when new file is created
in /dev, it will then jump in and relabel this file back to the
expected default context:
system_u:object_r:tun_tap_device_t:s0

There is a time gap happened.
Sometimes, it will have migration failed, AVC error message:
type=AVC msg=audit(1349858424.233:42507): avc:  denied  { read write } for
pid=19926 comm="qemu-kvm" path="/dev/tap33" dev=devtmpfs ino=131524
scontext=unconfined_u:system_r:svirt_t:s0:c598,c908
tcontext=system_u:object_r:device_t:s0 tclass=chr_file

This patch will label the tapfd device before qemu process starts:
system_u:object_r:tun_tap_device_t:MCS(MCS from seclabel->label)

This patch adds support for SUSPEND_DISK event; both lifecycle and
separated.  The support is added for QEMU, machines are changed to
PMSUSPENDED, but as QEMU sends SHUTDOWN afterwards, the state changes
to shut-off.  This and much more needs to be done in order for libvirt
to work with transient devices, wake-ups etc.  This patch is not
aiming for that functionality.

Commit e8fd8757 changed 'const char *'
category to virLogSource enum. This changes it in virLogEatParams as
well, thus fixing the build with --disable-debug.
--
Hopefully moving the enum declarations is less ugly than using int.

Upstream kernel introduced new sysfs knob "merge_across_nodes" to
specify if pages from different numa nodes can be merged. When set
to 0, only pages which physically reside in the memory area of
same NUMA node can be merged. When set to 1, pages from all nodes
can be merged.

This patch supports the tuning by adding new param field
"shm_merge_across_nodes".

This patch resolves:

  https://bugzilla.redhat.com/show_bug.cgi?id=805071

to the extent that it can be resolved with current qemu functionality.
It attempts to detect as many situations as possible when the simple
operation of disconnecting an existing tap device from one bridge and
attaching it to another will satisfy the change requested in
virDomainUpdateDeviceFlags() for a network device. Before this patch,
that situation could only be detected if the pre-change interface
*and* the post-change interface definition were both "type='bridge'".
After this patch, it can also be detected if the before or after
interfaces are any combination of type='bridge' and type='network'
(the networks can be <forward mode='nat|route|bridge'>, as long as
they use a Linux host bridge and not macvtap connections).

This extra effort is especially useful since the recent discovery that
a netdev_del+netdev_add combo (to reconnect the network device with
completely different hostside configuration) doesn't work properly
with current qemu (1.2) unless it is accompanied by the matching
device_del+device_add - see this mailing list message for details:

  http://lists.nongnu.org/archive/html/qemu-devel/2012-10/msg02355.html

(A slight modification of the patch referenced there has been prepared
to apply on top of this patch, but won't be pushed until qemu can be
made to work with it.)

* qemuDomainChangeNet needs access to the virDomainDeviceDef that
holds the new netdef (so that it can clear out the virDomainDeviceDef
if it ends up using the NetDef to replace the original), so the
virDomainNetDefPtr arg is replaced with a virDomainDeviceDefPtr.

* qemuDomainChangeNet previously checked for *some* changes to the
interface config, but this check was by no means complete. It was also
a bit disorganized.

This refactoring of the code is (I believe) complete in its check of
all NetDef attributes that might be changed, and either returns a
failure (for changes that are simply impossible), or sets one of three
flags:

  needLinkStateChange - if the device link state needs to go up/down
  needBridgeChange    - if everything else is the same, but it needs
                        to be connected to a difference linux host
                        bridge
  needReconnect       - if the entire host side of the device needs
                        to be torn down and reconstructed (currently
                        non-working, as mentioned above)

Note that this function will refuse to make any change that requires
the *guest* side of the device to be detached (e.g. changing the PCI
address or mac address). Those would be disruptive enough to the guest
that it's reasonable to require an explicit detach/attach sequence
from the management application.

* As mentioned above, qemuDomainChangeNet also does its best to
understand when a simple change in attached bridge for the existing
tap device will work vs. the need to completely tear down/reconstruct
the host side of the device (including tap device).

This patch *does not* implement the "reconnect" code anyway - there is
a placeholder that turns that into an error. Rather, the purpose of
this patch is to replicate existing behavior with code that is ready
to have that functionality plugged in in a later patch.

* The expanded uses for qemuDomainChangeNetBridge meant that it needed
to be enhanced as well - it no longer replaces the original brname
string in olddev with the new brname; instead, it relies on the
caller to replace the *entire* olddev with newdev (since we've gone
to great lengths to assure they are functionally identical other
than the name of the bridge, this is now not only safe, but more
correct). Additionally, qemuDomainNetChangeBridge can now set the
bridge for type='network' interfaces as well as plain type='bridge'
interfaces. (Note that I had to make this change simultaneous to the
reorganization of qemuDomainChangeNet because the two are too
closely intertwined to separate).

by using the same condition as for the <cpuset>.

Fixes "make check" found by
    http://honk.sigxcpu.org:8001/job/libvirt-check/160/

This does a shallow copy of all the bits, then strdups the two items
that are actually allocated separately.

This function really should have been taking virDevicePCIAddress*
instead of the inefficient virDevicePCIAddress (results in copying two
entire structs onto the stack rather than just two pointers), and
returning a bool true/false (not matching is not necessarily a
"failure", as a -1 return would imply, and also using "if
(!virDevicePCIAddressEqual(x, y))" to mean "if x == y" is just a bit
counterintuitive).

that broke "make syntax-check"

found by http://honk.sigxcpu.org:8001/job/libvirt-syntax-check/157/

Pushed under the build breaker rule.

If the vcpu placement is "static", it's just fine to ignore the
def->cpumask if emulatorpin is specified.

When vcpu placement is "auto", the domain process will be pinned
to advisory nodeset from querying numad, While emulatorpin will
override the pinning. That means both of them are to set the
pinning policy for domain process, but conflicts with each other.

This patch ingore emulatorpin if vcpu placement is "auto", because
<vcpu> placement can't be simply ignored for <numatune> placement
could default to it.

The onlined vcpu pinning policy should inherit def->cpuset if
it's not specified explicitly, and the affinity should be set
in this case. Oppositely, the offlined vcpu pinning policy should
be free()'ed.

Various APIs use cgroup to either set or get the statistics of
host or guest. Hotplug or hot unplug new vcpus without creating
or removing the cgroup for the vcpus could cause problems for
those APIs. E.g.

% virsh vcpucount dom
maximum      config        10
maximum      live          10
current      config         1
current      live           1

% virsh setvcpu dom 2

% virsh schedinfo dom --set vcpu_quota=1000
Scheduler      : posix
error: Unable to find vcpu cgroup for rhel6.2(vcpu: 1): No such file or
directory

This patch fixes the problem by creating cgroups for each of the
onlined vcpus, and destroying cgroups for each of the offlined
vcpus.

Document for <vcpu>'s "cpuset" says:

Since 0.4.4, this element can contain an optional cpuset attribute,
which is a comma-separated list of physical CPU numbers that virtual
CPUs can be pinned to.

However, it's not the truth, libvirt actually pins the domain
process to the specified pCPUs by "cpuset" of <vcpu>. And the
vcpu thread are pinned to all available pCPUs if no <vcpupin>
is specified for it.

This patch is to implement the codes to inherit <vcpu>'s "cpuset" for
vcpu that doesn't have <vcpupin> specified, and <vcpupin>
for these vcpu will be ignored when formating. Underlying
driver implementation will make sure the vcpu thread pinned
to correct pCPUs.

Setting pinning policy for vcpu which exceeds current vcpus number
just makes no sense, however, it could cause various problems, E.g.

<vcpu current='1'>4</vcpu>
<cputune>
  <vcpupin vcpuid='3' cpuset='4'/>
</cputune>

% virsh start linux
error: Failed to start domain linux
error: cannot set CPU affinity on process 32534: No such process

We must have some odd codes underlying which produces the
"on process 32534", but the point is why we not to prevent
earlier when parsing? Note that this is only one of the
problem it could cause.

This patch is to ignore the <vcpupin> for not onlined vcpus.

These 3 elements conflicts with each other in either the doc
or the underlying codes.

Current problems:

Problem 1:

The doc shouldn't simply say "These settings are superseded
by CPU tuning. " for element <vcpu>. As except the tuning, <vcpu>
allows to specify the current, maxmum vcpu number. Apart from that,
<vcpu> also allows to specify the placement as "auto", which binds
the domain process to the advisory nodeset from numad.

Problem 2:

Doc for <vcpu> says its "cpuset" specify the physical CPUs
that the vcpus can be pinned. But it's not the truth, as
actually it only pin domain process to the specified physical
CPUs. So either it's a document bug, or code bug.

Problem 3:

Doc for <vcpupin> says it supersed "cpuset" of <vcpu>, it's
not quite correct, as each <vcpupin> specify the pinning policy
only for one vcpu. How about the ones which doesn't have
<vcpupin> specified? it says the vcpu will be pinned to all
available physical CPUs, but what's the meaning of attribute
"cpuset" of <vcpu> then?

Problem 4:

Doc for <emulatorpin> says it pin the emulator threads (domain
process in other context, perhaps another follow up patch to
cleanup the inconsistency is needed) to the physical CPUs
specified its attribute "cpuset". Which conflicts with
<vcpu>'s "cpuset". And actually in the underlying codes,
it set the affinity for domain process twice if both
"cpuset" for <vcpu> and <emulatorpin> are specified,
and <emulatorpin>'s pinning will override <vcpu>'s.

Problem 5:

When "placement" of <vcpu> is "auto" (I.e. uses numad to
get the advisory nodeset to which the domain process is
pinned to), it will also be overridden by <emulatorpin>,

This patch is trying to sort out the conflicts or bugs by:

1) Don't say <vcpu> is superseded by <cputune>

2) Keep the semanteme for "cpuset" of <vcpu> (I.e. Still says it
   specify the physical CPUs the virtual CPUs). But modifying it
   to mention it also set the pinning policy for domain process,
   and the CPU placement of domain process specified by "cpuset"
   of <vcpu> will be ingored if <emulatorpin> specified, and
   similary, the CPU placement of vcpu thread will be ignored
   if it has <vcpupin> specified, for vcpu which doesn't have
   <vcpupin> specified, it inherits "cpuset" of <vcpu>.

3) Don't say <vcpu> is supersed by <vcpupin>. If neither <vcpupin>
   nor "cpuset" of <vcpu> is specified, the vcpu will be pinned
   to all available pCPUs.

4) If neither <emulatorpin> nor "cpuset" of <vcpu> is specified,
   the domain process (emulator threads in the context) will be
   pinned to all available pCPUs.

5) If "placement" of <vcpu> is "auto", <emulatorpin> is not allowed.

6) hotplugged vcpus will also inherit "cpuset" of <vcpu>

Codes changes according to above document changes:

1) Inherit def->cpumask for each vcpu which doesn't have <vcpupin>
   specified, during parsing.

2) ping the vcpu which doesn't have <vcpupin> specified to def->cpumask
   either by cgroup for sched_setaffinity(2), which is actually done
   by 1).

3) Error out if "placement" == "auto", and <emulatorpin> is specified.
   Otherwise, <emulatorpin> is honored, and "cpuset" of <cpuset> is
   ignored.

4) Setup cgroup for each hotplugged vcpu, and setup the pinning policy
   by either cgroup or sched_setaffinity(2).

5) Remove cgroup and <vcpupin> for each hot unplugged vcpu.

Patches are following (6 in total except this patch)

- Add the XML header so vim gives us syntax highlighting
- polkit-policy-file-validate hasn't existed for 3 years
- Permissions comment was not accurate

Just tweak it at build time depending on what polkit version we are
building for.

This fixes problems on platforms where sizeof(long) != sizeof(long long)
like ia32.

Libssh2 transport support was enabled lately but the spec file wasn't
updated to take this into account. This caused libvirt to be built
without libssh2 support in Red Hat based OSes.

We are currently able to work only with non-translated SELinux
contexts, but we are using functions that work with translated
contexts throughout the code.  This patch swaps all SELinux context
translation relative calls with their raw sisters to avoid parsing
problems.

The problems can be experienced with mcstrans for example.  The
difference is that if you have translations enabled (yum install
mcstrans; service mcstrans start), fgetfilecon_raw() will get you
something like 'system_u:object_r:virt_image_t:s0', whereas
fgetfilecon() will return 'system_u:object_r:virt_image_t:SystemLow'
that we cannot parse.

I was trying to confirm that the _raw variants were here since the dawn of
time, but the only thing I see now is that it was imported together in
the upstream repo [1] from svn, so before 2008.

Thanks Laurent Bigonville for finding this out.

[1] http://oss.tresys.com/git/selinux.git

When startupPolicy set for a USB devices allows such device to be
missing, there was no way this could be detected from domain XML. With
this patch, libvirt emits a new missing='yes' attribute for such devices
when active domain XML is generated.

SIGINT hasn't been blocked, which could lead to losing it somewhere in
virDomainGetBlockJobInfo and not aborting the job.