In order to optionally take advantage of new features in dnsmasq when
the host's version of dnsmasq supports them, but still be able to run
on hosts that don't support the new features, we need to be able to
detect the version of dnsmasq running on the host, and possibly
determine from the help output what options are in this dnsmasq.

This patch implements a greatly simplified version of the capabilities
code we already have for qemu. A dnsmasqCaps device can be created and
populated either from running a program on disk, reading a file with
the concatenated output of "dnsmasq --version; dnsmasq --help", or
examining a buffer in memory that contains the concatenated output of
those two commands. Simple functions to retrieve capabilities flags,
the version number, and the path of the binary are also included.

bridge_driver.c creates a single dnsmasqCaps object at driver startup,
and disposes of it at driver shutdown. Any time it must be used, the
dnsmasqCapsRefresh method is called - it checks the mtime of the
binary, and re-runs the checks if the binary has changed.

networkxml2argvtest.c creates 2 "artificial" dnsmasqCaps objects at
startup - one "restricted" (doesn't support --bind-dynamic) and one
"full" (does support --bind-dynamic). Some of the test cases use one
and some the other, to make sure both code pathes are tested.

Remove the obsolete 'qemud' naming prefix and underscore
based type name. Introduce virQEMUDriverPtr as the replacement,
in common with LXC driver naming style

This bug leads to getting incorrect vcpupin information via
qemudDomainGetVcpuPinInfo() API when the number of maximum
cpu on a host falls into a range such as 31 < ncpus < 64.

gcc warning:
left shift count >= width of type

The following bug is such the case
https://bugzilla.redhat.com/show_bug.cgi?id=876415

Signed-off-by: NHarsh Prateek Bora <harsh@linux.vnet.ibm.com>

bridge_driver.h: silence gcc warnings:
statement with no effect [-Wunused-value]
unused variable 'net' [-Wunused-variable]

virdrivermoduletest.c: don't require network driver module
if it hasn't been built.

I was convicted that space at EOL should no be there
even for qemu help data. Hence, I've removed one in
commit bb2f6216. However, it turns out we want
it exactly the way qemu produces it. So I should undo
my premature fix. A patch against qemu has been posted
as well.

Both generated with
qemu-system-x86_64 --help > qemu-1.2.0

qemu-system-x86_64 \
-device ? \
-device pci-assign,? \
-device virtio-blk-pci,? \
-device virtio-net-pci,? \
-device scsi-disk,? \
-device PIIX4_PM,? \
-device usb-redir,? \
-device ide-drive,? \
-device usb-host,? 2> qemu-1.2.0-device

It seems I missed a few -device flags when doing this last time and I
mixed up qemu and qemu-kvm.

... and update all users.  No change in functionality, the parameter
will be used in the next patch.
Signed-off-by: NMiloslav Trmač <mitr@redhat.com>

qemumonitorjsontest creates a temporary directory to hold the socket
that is simulating the monitor socket. The directory containing the
socket wasn't disposed properly at the end of the test leaving garbage
in the temporary folder.

When doing the qemumonitorjsontest on a machine under heavy load the
test tends to deadlock from time to time. This patch adds the hack to
break the event loop that is used in virsh.

The AMD Bulldozer architecture uses so called "Clustered integer core
modules" that count both as threads and cores. This patch expects the
cpu to be detected using the new fallback condition otherwise twice the
number of processors would be detected.

This test data was gathered on an AMD MagnyCours machine that reports it
has only one NUMA node although the hardware is consisting of 4. As
duplicate core id's are ignored the reported topology was bogous. This
should be fixed by the previous patch.

Reported and data provided by George-Cristian Bîrzan.

For S390, the default console target type cannot be of type 'serial'.
It is necessary to at least interpret the 'arch' attribute
value of the os/type element to produce the correct default type.

Therefore we need to extend the signature of defaultConsoleTargetType
to account for architecture. As a consequence all the drivers
supporting this capability function must be updated.

Despite the amount of changed files, the only change in behavior is
that for S390 the default console target type will be 'virtio'.

N.B.: A more future-proof approach could be to to use hypervisor
specific capabilities to determine the best possible console type.
For instance one could add an opaque private data pointer to the
virCaps structure (in case of QEMU to hold capsCache) which could
then be passed to the defaultConsoleTargetType callback to determine
the console target type.
Seems to be however a bit overengineered for the use case...
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

External checkpoints could be created with snapshot-create, but
without libvirt supplying a default name for the memory file,
it is essential to add a new argument to snapshot-create-as to
allow the user to choose the memory file name.  This adds the
option --memspec [file=]name[,snapshot=type], where type can
be none, internal, or external.  For an example,

virsh snapshot-create-as $dom --memspec /path/to/file

is the shortest possible command line for creating an external
checkpoint, named after the current timestamp.

* tools/virsh-snapshot.c (vshParseSnapshotMemspec): New function.
(cmdSnapshotCreateAs): Use it.
* tests/virsh-optparse (test_url): Test it.
* tools/virsh.pod (snapshot-create-as): Document it.

Each <domainsnapshot> can now contain an optional <memory>
element that describes how the VM state was handled, similar
to disk snapshots.  The new element will always appear in
output; for back-compat, an input that lacks the element will
assume 'no' or 'internal' according to the domain state.

Along with this change, it is now possible to pass <disks> in
the XML for an offline snapshot; this also needs to be wired up
in a future patch, to make it possible to choose internal vs.
external on a per-disk basis for each disk in an offline domain.
At that point, using the --disk-only flag for an offline domain
will be able to work.

For some examples below, remember that qemu supports the
following snapshot actions:

qemu-img: offline external and internal disk
savevm: online internal VM and disk
migrate: online external VM
transaction: online external disk

=====
<domainsnapshot>
  <memory snapshot='no'/>
  ...
</domainsnapshot>

implies that there is no VM state saved (mandatory for
offline and disk-only snapshots, not possible otherwise);
using qemu-img for offline domains and transaction for online.

=====
<domainsnapshot>
  <memory snapshot='internal'/>
  ...
</domainsnapshot>

state is saved inside one of the disks (as in qemu's 'savevm'
system checkpoint implementation).  If needed in the future,
we can also add an attribute pointing out _which_ disk saved
the internal state; maybe disk='vda'.

=====
<domainsnapshot>
  <memory snapshot='external' file='/path/to/state'/>
  ...
</domainsnapshot>

This is not wired up yet, but future patches will allow this to
control a combination of 'virsh save /path/to/state' plus disk
snapshots from the same point in time.

=====

So for 1.0.1 (and later, as needed), I plan to implement this table
of combinations, with '*' designating new code and '+' designating
existing code reached through new combinations of xml and/or the
existing DISK_ONLY flag:

domain  memory  disk   disk-only | result
-----------------------------------------
offline omit    omit   any       | memory=no disk=int, via qemu-img
offline no      omit   any       |+memory=no disk=int, via qemu-img
offline omit/no no     any       | invalid combination (nothing to snapshot)
offline omit/no int    any       |+memory=no disk=int, via qemu-img
offline omit/no ext    any       |*memory=no disk=ext, via qemu-img
offline int/ext any    any       | invalid combination (no memory to save)
online  omit    omit   off       | memory=int disk=int, via savevm
online  omit    omit   on        | memory=no disk=default, via transaction
online  omit    no/ext off       | unsupported for now
online  omit    no     on        | invalid combination (nothing to snapshot)
online  omit    ext    on        | memory=no disk=ext, via transaction
online  omit    int    off       |+memory=int disk=int, via savevm
online  omit    int    on        | unsupported for now
online  no      omit   any       |+memory=no disk=default, via transaction
online  no      no     any       | invalid combination (nothing to snapshot)
online  no      int    any       | unsupported for now
online  no      ext    any       |+memory=no disk=ext, via transaction
online  int/ext any    on        | invalid combination (disk-only vs. memory)
online  int     omit   off       |+memory=int disk=int, via savevm
online  int     no/ext off       | unsupported for now
online  int     int    off       |+memory=int disk=int, via savevm
online  ext     omit   off       |*memory=ext disk=default, via migrate+trans
online  ext     no     off       |+memory=ext disk=no, via migrate
online  ext     int    off       | unsupported for now
online  ext     ext    off       |*memory=ext disk=ext, via migrate+transaction

* docs/schemas/domainsnapshot.rng (memory): New RNG element.
* docs/formatsnapshot.html.in: Document it.
* src/conf/snapshot_conf.h (virDomainSnapshotDef): New fields.
* src/conf/domain_conf.c (virDomainSnapshotDefFree)
(virDomainSnapshotDefParseString, virDomainSnapshotDefFormat):
Manage new fields.
* tests/domainsnapshotxml2xmltest.c: New test.
* tests/domainsnapshotxml2xmlin/*.xml: Update existing tests.
* tests/domainsnapshotxml2xmlout/*.xml: Likewise.

The libvirt coding standard is to use 'function(...args...)'
instead of 'function (...args...)'. A non-trivial number of
places did not follow this rule and are fixed in this patch.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

to avoid ENAMETOOLONG:

https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=amd64&ver=1.0.0~rc1-1&stamp=1351453521

The test data contained extra \0 bytes after newlines probably due to a
kernel off-by-one bug.

Sometimes it's handy to know how many bits are set.

* src/util/bitmap.h (virBitmapCountBits): New prototype.
(virBitmapNextSetBit): Use correct type.
* src/util/bitmap.c (virBitmapNextSetBit): Likewise.
(virBitmapSetAll): Maintain invariant of clear tail bits.
(virBitmapCountBits): New function.
* src/libvirt_private.syms (bitmap.h): Export it.
* tests/virbitmaptest.c (test2): Test it.

Currently it's assumed that qemu always supports VNC, however it is
definitely possible to compile qemu without VNC support so we should at
the very least check for it and handle that correctly.

Several tests assume that VNC is always available and include it in
their configs and the expected command line. The tests have nothing to
do with graphics display so they shouldn't rely on VNC.

This fixes the problem reported in:

  https://bugzilla.redhat.com/show_bug.cgi?id=868389

Previously, the dnsmasq hosts file (used for static dhcp entries, and
addnhosts file (used for additional dns host entries) were only
created/referenced on the dnsmasq commandline if there was something
to put in them at the time the network was started. Once we can update
a network definition while it's active (which is now possible with
virNetworkUpdate), this is no longer a valid strategy - if there were
0 dhcp static hosts (resulting in no reference to the hosts file on the
commandline), then one was later added, the commandline wouldn't have
linked dnsmasq up to the file, so even though we create it, dnsmasq
doesn't pay any attention.

The solution is to just always create these files and reference them
on the dnsmasq commandline (almost always, anyway). That way dnsmasq
can notice when a new entry is added at runtime (a SIGHUP is sent to
dnsmasq by virNetworkUdpate whenever a host entry is added or removed)

The exception to this is that the dhcp static hosts file isn't created
if there are no lease ranges *and* no static hosts. This is because in
this case dnsmasq won't be setup to listen for dhcp requests anyway -
in that case, if the count of dhcp hosts goes from 0 to 1, dnsmasq
will need to be restarted anyway (to get it listening on the dhcp
port). Likewise, if the dhcp hosts count goes from 1 to 0 (and there
are no dhcp ranges) we need to restart dnsmasq so that it will stop
listening on port 67. These special situations are handled in the
bridge driver's networkUpdate() by checking for ((bool)
nranges||nhosts) both before and after the update, and triggering a
dnsmasq restart if the before and after don't match.

We have historically allowed 'aio' as a synonym for 'raw' for
back-compat to xen, but since a future patch will move to using
an enum value, we have to pick one to be our preferred output
name.  This is a slight change in the output XML, but the sexpr
and xm outputs should still be identical, and the input XML can
still use either form.

* src/conf/domain_conf.c (virDomainDiskDefForeachPath): Move aio
back-compat...
(virDomainDiskDefParseXML): ...to parse time.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenFormatSxprDisk): ...and
to output time.
* src/xenxs/xen_xm.c (xenParseXM, xenFormatXMDisk): Likewise.
* tests/sexpr2xmldata/sexpr2xml-*.xml: Update tests.

This patch adds QEMU support for the "relaxed" feature implemented by
previous patch.

The previously introduced virFile{Lock,Unlock} APIs provide a
way to acquire/release fcntl() locks on individual files. For
unknown reason though, the POSIX spec says that fcntl() locks
are released when *any* file handle referring to the same path
is closed. In the following sequence

  threadA: fd1 = open("foo")
  threadB: fd2 = open("foo")
  threadA: virFileLock(fd1)
  threadB: virFileLock(fd2)
  threadB: close(fd2)

you'd expect threadA to come out holding a lock on 'foo', and
indeed it does hold a lock for a very short time. Unfortunately
when threadB does close(fd2) this releases the lock associated
with fd1. For the current libvirt use case for virFileLock -
pidfiles - this doesn't matter since the lock is acquired
at startup while single threaded an never released until
exit.

To provide a more generally useful API though, it is necessary
to introduce a slightly higher level abstraction, which is to
be referred to as a "lockspace".  This is to be provided by
a virLockSpacePtr object in src/util/virlockspace.{c,h}. The
core idea is that the lockspace keeps track of what files are
already open+locked. This means that when a 2nd thread comes
along and tries to acquire a lock, it doesn't end up opening
and closing a new FD. The lockspace just checks the current
list of held locks and immediately returns VIR_ERR_RESOURCE_BUSY.

NB, the API as it stands is designed on the basis that the
files being locked are not being otherwise opened and used
by the application code. One approach to using this API is to
acquire locks based on a hash of the filepath.

eg to lock /var/lib/libvirt/images/foo.img the application
might do

   virLockSpacePtr lockspace = virLockSpaceNew("/var/lib/libvirt/imagelocks");
   lockname = md5sum("/var/lib/libvirt/images/foo.img");
   virLockSpaceAcquireLock(lockspace, lockname);

NB, in this example, the caller should ensure that the path
is canonicalized before calculating the checksum.

It is also possible to do locks directly on resources by
using a NULL lockspace directory and then using the file
path as the lock name eg

   virLockSpacePtr lockspace = virLockSpaceNew(NULL);
   virLockSpaceAcquireLock(lockspace, "/var/lib/libvirt/images/foo.img");

This is only safe to do though if no other part of the process
will be opening the files. This will be the case when this
code is used inside the soon-to-be-reposted virlockd daemon
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add test for 280b8c9e.

We are currently able to work only with non-translated SELinux
contexts, but we are using functions that work with translated
contexts throughout the code.  This patch swaps all SELinux context
translation relative calls with their raw sisters to avoid parsing
problems.

The problems can be experienced with mcstrans for example.  The
difference is that if you have translations enabled (yum install
mcstrans; service mcstrans start), fgetfilecon_raw() will get you
something like 'system_u:object_r:virt_image_t:s0', whereas
fgetfilecon() will return 'system_u:object_r:virt_image_t:SystemLow'
that we cannot parse.

I was trying to confirm that the _raw variants were here since the dawn of
time, but the only thing I see now is that it was imported together in
the upstream repo [1] from svn, so before 2008.

Thanks Laurent Bigonville for finding this out.

[1] http://oss.tresys.com/git/selinux.git

With the recent introduction of QMP capabilities probing, libvirt failed
to detect support for QXL graphics in QEMU 1.2 and newer. In addition to
fixing that, this patch also causes libvirt to detect QXL support for
qemu-kvm-0.13.0, which doesn't advertise it in -help output but mentions
it in device list. Since qemu-kvm-0.13.0 supported -spice, it looks like
not having qxl in -help was a bug.

When both kvmclock and kvm_pv_eoi are configured (either disabled or
enabled) libvirt will generate invalid CPU specification due to the
fact that even though kvmclock causes the CPU to be specified, it
doesn't set have_cpu flag to true (and the new kvm_pv_eoi as well).
This patch fixes the issue and adds a test exactly for that to show
that it is fixed correctly (and also to keep it that way in the future
of course).

Currently the qemuCapsParseDeviceStr method has a bunch of open
coded string searches/comparisons to detect devices and their
properties. Soon this data will be obtained from QMP queries
instead of -device help output. Maintaining the list of device
and properties in two places is undesirable. Thus the existing
qemuCapsParseDeviceStr() method needs to be refactored to
separate the device types and properties from the actual
search code.

Thus the -device help output is now parsed to construct a
list of device names, and device properties. These are then
checked against a set of datatables to set the capability
flags
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The 'const char *category' parameter only has a few possible
values now that the filename has been separated. Turn this
parameter into an enum instead.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the logging APIs have a 'const char *category' parameter
which indicates where the log message comes from. This is typically
a combination of the __FILE__ string and other prefix. Split the
__FILE__ off into a dedicated parameter so it can passed to the
log outputs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The log destinations are an enum, but most of the code was
just using a plain 'int' for function params / variables.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The __LINE__ macro value is specified to fit in the size_t
type, so use that instead of 'long long' in the logging code
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The log priority levels are an enum, but most of the code was
just using a plain 'int' for function params / variables.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In addition to the preformatted text line, pass the raw message as well,
to allow the output functions to use a different output format.
Signed-off-by: NMiloslav Trmač <mitr@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>