Replace the inline "auth" struct in virStorageSource with a pointer
to a virStorageAuthDefPtr and utilize between the domain_conf, qemu_conf,
and qemu_command sources for finding the auth data for a domain disk

Use the probing functionality added in the last patch to turn on
a capability bit when active commit is present, and gate active
commit on that capability.

For my own reference: the difference between BLOCKJOB_SYNC and
BLOCKJOB_ASYNC is whether qemu generated an event at the
conclusion of blockpull; basically, RHEL 6.2 was the only release
of qemu that has the sync semantics and lacks the event.  RHEL
6.3 added blockcopy, but also picked up on the upstream style
of qemu generating events.  As no one is likely to backport
active commit to RHEL 6.2, it's safe for blockcommit to always
require async blockjob support.

Modifying qemucapabilitiestest is painful; the .replies files would
be so much easier if they had comments correlating which command
generated the given reply.  Maybe I'll fix that up later...

* src/qemu/qemu_capabilities.h (QEMU_CAPS_ACTIVE_COMMIT): New
capability.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Use the new bit
* src/qemu/qemu_capabilities.c (virQEMUCaps): Name the new bit.
(virQEMUCapsProbeQMPCommands): Set it.
* tests/qemucapabilitiesdata/caps_1.3.1-1.replies: Update.
* tests/qemucapabilitiesdata/caps_1.4.2-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.5.3-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.6.0-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.6.50-1.replies: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

We are about to turn on support for active block commit.  Although
qemu 2.0 was the first version to mostly support it, that version
mis-handles 0-length files, and doesn't have anything available for
easy probing.  But qemu 2.1 fixed bugs, and made life simpler by
letting the 'top' argument be optional.  Unless someone begs for
active commit with qemu 2.0, for now we are just going to enable
it only by probing for qemu 2.1 behavior (anyone backporting active
commit can also backport the optional argument behavior).  This
requires qemu.git commit 7676e2c597000eff3a7233b40cca768b358f9bc9.

Although all our actual uses of block-commit supply arguments for
both base and top, we can omit both arguments and use a bogus
device string to trigger an interesting behavior in qemu.  All QMP
commands first do argument validation, failing with GenericError
if a mandatory argument is missing.  Once that passes, the code
in the specific command gets to do further checking, and the qemu
developers made sure that if device is the only supplied argument,
then the block-commit code will look up the device first, with a
failure of DeviceNotFound, before attempting any further argument
validation (most other validations fail with GenericError).  Thus,
the category of error class can reliably be used to decipher
whether the top argument was optional, which in turn implies a
working active commit.  Since we expect our bogus device string to
trigger an error either way, the code is written to return a
distinct return value without spamming the logs.

* src/qemu/qemu_monitor.h (qemuMonitorSupportsActiveCommit): New
prototype.
* src/qemu/qemu_monitor.c (qemuMonitorSupportsActiveCommit):
Implement it.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONBlockCommit):
Allow NULL for top and base, for probing purposes.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONBlockCommit):
Likewise, implementing the probe.
* tests/qemumonitorjsontest.c (mymain): Enable...
(testQemuMonitorJSONqemuMonitorSupportsActiveCommit): ...a new test.
Signed-off-by: NEric Blake <eblake@redhat.com>

So far only information on disks and host devices are exposed in the
capabilities XML. Well, at least something. Even a new test is
introduced. The qemu capabilities are stolen from already existing
qemucapabilities test. There's one tricky point though. Functions that
checks host's KVM and VFIO capabilities, are impossible to mock
currently. So in the test, we are setting the capabilities by hand.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Sometimes it may be useful to get a default machine for given qemu
binary. Fortunately, the default machine is stored always on the first
position in the supported machines array.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This internal API is meant to answer the question 'Is this machine
type supported by given qemu?'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The API may come handy if somebody has an architecture and wants to
look through available qemus if the architecture is supported or not.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The functions called here report an OOM error when the allocation
fails, or quietly return -1 on wrong usage (which is not the case
here)

They report errors in all other cases.

Replace:
if (virBufferError(&buf)) {
    virBufferFreeAndReset(&buf);
    virReportOOMError();
    ...
}

with:
if (virBufferCheckError(&buf) < 0)
    ...

This should not be a functional change (unless some callers
misused the virBuffer APIs - a different error would be reported
then)

So far, we only report an error if formatting the siblings bitmap
in NUMA topology fails.

Be consistent and always report error in virCapabilitiesFormatXML.

https://bugzilla.redhat.com/show_bug.cgi?id=1086121

We now support startupPolicy='optional' for disks, but this
should work only for cold boot, not for restore or migrate.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

My commit fba6bc47 iterated over the features in cpu definition
without checking if there is one.

This introduces two new attributes "cmd_per_lun" and "max_sectors" same
with the names QEMU uses for virtio-scsi. An example of the XML:

<controller type='scsi' index='0' model='virtio-scsi' cmd_per_lun='50'
max_sectors='512'/>

The corresponding QEMU command line:

-device virtio-scsi-pci,id=scsi0,cmd_per_lun=50,max_sectors=512,
bus=pci.0,addr=0x3
Signed-off-by: NMike Perez <thingee@gmail.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Add suport for invariant TSC flag (CPUID 0x80000007, bit 8 of EDX).
If this flag is enabled, the TSC ticks at a constant rate across
all ACPI P-, C- and T-states.

This can be enabled by adding:
<feature name='invtsc'/>
to the <cpu> element.

Migration and saving the domain does not work with this flag.

QEMU support: http://git.qemu.org/?p=qemu.git;a=commitdiff;h=303752a

The feature name "invtsc" differs from the name "" used by the linux kernel:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/cpu/powerflags.c?id=30321c7b#n18

The IDE bus doesn't support readonly disks, so inform the user with an
error message instead of let qemu fail with a more obscure "Device
'ide-hd' could not be initialized" error message.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1112939Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

We have the following matrix of possible arguments handled by the logic
statement touched by this patch:
       | flags & _REUSE_EXT | !(flags & _REUSE_EXT)
-------+--------------------+----------------------
 format| (1)                | (2)
-------+--------------------+----------------------
!format| (3)                | (4)
-------+--------------------+----------------------

In cases 1 and 2 the user provided a format, in cases 3 and 4 not. The
user requests to use a pre-existing image in 1 and 3 and libvirt will
create a new image in 2 and 4.

The difference between cases 3 and 4 is that for 3 the format is probed
from the user-provided image, whereas in 4 we just use the existing disk
format.

The current code would treat cases 1,3 and 4 correctly but in case 2 the
format provided by the user would be ignored.

The particular piece of code was broken in commit 35c7701c
but since it was introduced a few commits before that it was never
released as working.

Commit 55bbb011 introduced a regression
where we forgot to save the persistent domain configuration after an
external snapshot. This would make libvirt forget the snapshots and
effectively revert to the previous state in the following scenario:

1) Start VM
2) Take snapshot
3) Destroy VM
4) Restart libvirtd

Also fix spurious blank line added by patch mentioned above.

Since commit d86c876a we are using
guestfwd=tcp:IP:PORT,chardev=ID for guestfwd specification, however,
that has not changed in qemu, so guestfwd does not work since.

Apart from that, guestfwd is not working with older qemu that doesn't
have QEMU_CAPS_DEVICE.

Both regressions exist since late 2009 and nobody found that (until
now), so I'm only fixing the first one.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1112066Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The QEMU VNC client arg code has a long standing typo
of SASL_CONF_DIR when it should be SASL_CONF_PATH for
the env variable name.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When creating a new disk mirror the new struct is stored in a separate
variable until everything went well. The removed hunk would actually
remove existing mirror information for example when the api would be run
if a mirror still exists.

I'm going to add functions that will deal with individual image files
rather than whole disks. Rename the security function to make room for
the new one.

Rename them to comply with the naming policy.

The new VIR_CONNECT_COMPARE_CPU_FAIL_INCOMPATIBLE flag for
virConnectCompareCPU can be used to get an error
(VIR_ERR_CPU_INCOMPATIBLE) describing the incompatibility instead of the
usual VIR_CPU_COMPARE_INCOMPATIBLE return code.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

When CPU comparison APIs return VIR_CPU_COMPARE_INCOMPATIBLE, the caller
has no clue why the CPU is considered incompatible with host CPU. And in
some cases, it would be nice to be able to get such info in a client
rather than having to look in logs.

To achieve this, the APIs can be told to return VIR_ERR_CPU_INCOMPATIBLE
error for incompatible CPUs and the reason will be described in the
associated error message.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Currently, only LXC has hostdev mode 'capabilities' support,
so the other drivers should forbid to define it in XML.
The hostdev mode check is added to devicesPostParseCallback()
for each hypervisor driver.

But there are some drivers lack function devicesPostParseCallback(),
so only add check for qemu, libxl, openvz, uml, xen, xenapi.
Signed-off-by: NJincheng Miao <jmiao@redhat.com>

As we are doing with the enum structures, a cleanup in "src/qemu/"
directory was done now. All the enums that were defined in the
header files were converted to typedefs in this directory. This
patch includes all the adjustments to remove conflicts when you do
this kind of change. "Enum-to-typedef"'s conversions were made in
"src/qemu/qemu_{capabilities, domain, migration, hotplug}.h".
Signed-off-by: NJulio Faracco <jcfaracco@gmail.com>

When looking for a port to allocate, the port allocator didn't take in
consideration ports that are statically set by the user.  Defining
these two graphics elements in the XML would cause an error, as the
port allocator would try to use the same port for the spice graphics
element:

    <graphics type='spice' autoport='yes'/>
    <graphics type='vnc' port='5900' autoport='no'/>

The new *[pP]ortReserved variables keep track of the ports that were
successfully tracked as used by the port allocator but that weren't
bound.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1081881Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

There are no options to parse here other than the name of the device,
and all three possible device names have the same prefix
("virtio-balloon" with "-ccw", "-pci", or "-device" appended), so the
code is fairly simple. It has been implemented such that it will be
easier to add handling for other -device entries that aren't otherwise
recognized - just add another "else if (STRPREFIX(opts, ....)" clause.

qemuParseCommandLineString() previously would always add a <memballoon
model='virtio'/> to every result (the comments erroneously say that it
is adding a <memballoon model='none'/>) This has been changed to add
model='none', and 84 test case xml's updated accordingly (so that
qemuxml2argvtest won't fail).

Now that the memballoon device is properly parsed, we can safely add a
test for properly ignoring -nodefconfig and -nodefaults. Rather than
adding an entire new test case for this (and memballoon), we just
randomly pick the clock-utc test and modify it slightly to fulfill the
purpose.

Just code movement and rename.

I'm going to add functions that will deal with individual image files
rather than whole disks. Rename the security function to make room for
the new one.

Also remove one spurious ATTRIBUTE_UNUSED guarding the @migrated
argument.

The image labels are stored in the virStorageSource struct. Convert the
virDomainDiskDefGetSecurityLabelDef helper not to use the full disk def
and move it appropriately.

And add stubs to other drivers like: lxc, qemu, uml and vbox.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

There are two places where you'll find info on page sizes. The first
one is under <cpu/> element, where all supported pages sizes are
listed. Then the second one is under each <cell/> element which refers
to concrete NUMA node. At this place, the size of page's pool is
reported. So the capabilities XML looks something like this:

<capabilities>

  <host>
    <uuid>01281cda-f352-cb11-a9db-e905fe22010c</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Westmere</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='1' threads='1'/>
      ...
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>
    ...
    <topology>
      <cells num='4'>
        <cell id='0'>
          <memory unit='KiB'>4054408</memory>
          <pages unit='KiB' size='4'>1013602</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
          </cpus>
        </cell>
        <cell id='1'>
          <memory unit='KiB'>4071072</memory>
          <pages unit='KiB' size='4'>1017768</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='1' socket_id='0' core_id='0' siblings='1'/>
          </cpus>
        </cell>
        ...
      </cells>
    </topology>
    ...
  </host>

  <guest/>

</capabilities>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

For future work we want to get info for not only the free memory
but overall memory size too. That's why the function must have
new signature too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

When the block job event was first added, it was for block pull,
where the active layer of the disk remains the same name.  It was
also in a day where we only cared about local files, and so we
always had a canonical absolute file name.  But two things have
changed since then: we now have network disks, where determining
a single absolute string does not really make sense; and we have
two-phase jobs (copy and active commit) where the name of the
active layer changes between the first event (ready, on the old
name) and second (complete, on the pivoted name).

Adam Litke reported that having an unstable string between events
makes life harder for clients.  Furthermore, all of our API that
operate on a particular disk of a domain accept multiple strings:
not only the absolute name of the active layer, but also the
destination device name (such as 'vda').  As this latter name is
stable, even for network sources, it serves as a better string
to supply in block job events.

But backwards-compatibility demands that we should not change the
name handed to users unless they explicitly request it.  Therefore,
this patch adds a new event, BLOCK_JOB_2 (alas, I couldn't think of
any nicer name - but at least Migrate2 and Migrate3 are precedent
for a number suffix).  We must double up on emitting both old-style
and new-style events according to what clients have registered for
(see also how IOError and IOErrorReason emits double events, but
there the difference was a larger struct rather than changed
meaning of one of the struct members).

Unfortunately, adding a new event isn't something that can easily
be broken into pieces, so the commit is rather large.

* include/libvirt/libvirt.h.in (virDomainEventID): Add a new id
for VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2.
(virConnectDomainEventBlockJobCallback): Document new semantics.
* src/conf/domain_event.c (_virDomainEventBlockJob): Rename field,
to ensure we catch all clients.
(virDomainEventBlockJobNew): Add parameter.
(virDomainEventBlockJobDispose)
(virDomainEventBlockJobNewFromObj)
(virDomainEventBlockJobNewFromDom)
(virDomainEventDispatchDefaultFunc): Adjust clients.
(virDomainEventBlockJob2NewFromObj)
(virDomainEventBlockJob2NewFromDom): New functions.
* src/conf/domain_event.h: Add new prototypes.
* src/libvirt_private.syms (domain_event.h): Export new functions.
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Generate two
different events.
* src/qemu/qemu_process.c (qemuProcessHandleBlockJob): Likewise.
* src/remote/remote_protocol.x
(remote_domain_event_block_job_2_msg): New struct.
(REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2): New RPC.
* src/remote/remote_driver.c
(remoteDomainBuildEventBlockJob2): New handler.
(remoteEvents): Register new event.
* daemon/remote.c (remoteRelayDomainEventBlockJob2): New handler.
(domainEventCallbacks): Register new event.
* tools/virsh-domain.c (vshEventCallbacks): Likewise.
(vshEventBlockJobPrint): Adjust client.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

The block commit code looks for an explicit base file relative
to the discovered top file; so for a chain of:
  base <- snap1 <- snap2 <- snap3
and a command of:
  virsh blockcommit $dom vda --base snap2 --top snap1
we got a sane message (here from libvirt 1.0.5):
error: invalid argument: could not find base 'snap2' below 'snap1' in chain for 'vda'

Meanwhile, recent refactoring has slightly reduced the quality of the
libvirt error messages, by losing the phrase 'below xyz':
error: invalid argument: could not find image 'snap2' in chain for 'snap3'

But we had a one-off, where we were not excluding the top file
itself in searching for the base; thankfully qemu still reports
the error, but the quality is worse:
  virsh blockcommit $dom vda --base snap2 --top snap2
error: internal error unable to execute QEMU command 'block-commit': Base '/snap2' not found

Fix the one-off in blockcommit by changing the semantics of name
lookup - if a starting point is specified, then the result must
be below that point, rather than including that point.  The only
other call to chain lookup was blockpull code, which was already
forcing the lookup to omit the active layer and only needs a
tweak to use the new semantics.

This also fixes the bug exposed in the testsuite, where when doing
a lookup pinned to an intermediate point in the chain, we were
unable to return the name of the parent also in the chain.

* src/util/virstoragefile.c (virStorageFileChainLookup): Change
semantics for non-NULL startFrom.
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Adjust caller,
to keep existing semantics.
* tests/virstoragetest.c (mymain): Adjust to expose new semantics.
Signed-off-by: NEric Blake <eblake@redhat.com>

For block devices used as snapshot source the new snapshot code would
set the reuse flag. This inhibits to take snapshot without specially
preparing the block image before taking the snapshot.

Fortunately this is not a regression as only the new way of specifying
snapshot source is affected.

For the followin snapshot XML:
 <domainsnapshot>
   <disks>
     <disk name='vda' type='block'>
       <driver type='qcow2'/>
       <source dev="/dev/andariel/testsnap" />
     </disk>
   </disks>
 </domainsnapshot>

You'd get:
error: internal error: unable to execute QEMU command 'transaction': Image is not in qcow2 format

After this patch the snapshot is created successfully.