The current qemu capabilities are stored in a virBitmapPtr
object, whose type is exposed to callers. We want to store
more data besides just the flags, so we need to move to a
struct type. This object will also need to be reference
counted, since we'll be maintaining a cache of data per
binary. This change introduces a 'qemuCapsPtr' virObject
class. Most of the change is just renaming types and
variables in all the callers
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If no private data needs to be maintained, it can be useful
to create virDomainObjPtr instances without having a virCapsPtr
instance around. Adapt the virDomainObjNew() function to allow
for a NULL caps
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Technically speaking we should wait until we receive the QMP
greeting message before attempting to send any QMP monitor
commands. Mostly we've got away with this, but there is a race
in some QEMU which cause it to SEGV if you sent it data too
soon after startup. Waiting for the QMP greeting avoids the
race
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add an API allowing flags from one virBitmapPtr to be copied
into another instance.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Input XML snip:
<redirdev bus='usb' type='spicevmc'>
   <address type='usb' bus='0' port='4'/>
 </redirdev>
<redirfilter>
  <usbdev class='0x08' vendor='0x1234' product='0xbeef' \
          version='2.00' allow='yes'/>
  <usbdev class='-1' vendor='-1' product='-1' version='-1' allow='no'/>
</redirfilter>

will be converted to:
-device usb-redir,chardev=charredir0,id=redir0,\
filter=0x08:0x1234:0xBEEF:0x2000:1|-1:-1:-1:-1:0,bus=usb.0,port=4

https://bugzilla.redhat.com/show_bug.cgi?id=795929
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=6af165892cf900291046f1d25f95416f379504c2

This patch define and parse the input XML of USB redirection filter.
<devices>
...
  <redirdev bus='usb' type='spicevmc'>
    <address type='usb' bus='0' port='4'/>
  </redirdev>
  <redirfilter>
    <usbdev class='0x08' vendor='0x1234' product='0xbeef' \
            version='2.00' allow='yes'/>
    <usbdev allow='no'/>
  </redirfilter>
...
</devices>

There is no 1:1 mapping between ports and redirected devices and
qemu and spicy client couldn't decide into which usbredir ports
the client can 'plug' redirected devices. So it make sense to apply
all of filter rules global to all existing usb redirection devices.
class attribute is USB Class codes. version is bcdDevice value
of USB device. vendor and product is USB vendorId and productId.
-1 can be used to allow any value for a field. Except allow attribute
the other four are optional, default value is -1.

Add a qemu flag for USB redirection filter support.

The output:
usb-redir.chardev=chr
usb-redir.debug=uint8
usb-redir.filter=string
usb-redir.port=string

I got an off-list report about a bad diagnostic:
Target network card mac 52:54:00:49:07:ccdoes not match source 52:54:00:49:07:b8

True to form, I've added a syntax check rule to prevent it
from recurring, and found several other offenders.

* cfg.mk (sc_require_whitespace_in_translation): New rule.
* src/conf/domain_conf.c (virDomainNetDefCheckABIStability): Add
space.
* src/esx/esx_util.c (esxUtil_ParseUri): Likewise.
* src/qemu/qemu_command.c (qemuCollectPCIAddress): Likewise.
* src/qemu/qemu_driver.c (qemuDomainSetMetadata)
(qemuDomainGetMetadata): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeNetBridge): Likewise.
* src/rpc/virnettlscontext.c
(virNetTLSContextCheckCertDNWhitelist): Likewise.
* src/vmware/vmware_driver.c (vmwareDomainResume): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc, vboxAttachDrives):
Avoid false negatives.
* tools/virsh-domain.c (info_save_image_dumpxml): Reword.
Based on a report by Luwen Su.

Currently qemuMonitorOpen() requires an address of the QEMU
monitor. When doing QMP based capabilities detection it is
easier if a pre-opened FD can be provided, since then the
monitor can be run on the STDIO console. Add a new API
qemuMonitorOpenFD() for such usage
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

virDomainDefParseString assigns 0 to port if autoport enabled.
So fix code, which check different between old and new
configurations.

Don't try to dereferece NULL pointer.

Historically, the first <console> element is treated as the
alias of a <serial> device. In the virDomainDeviceInfoIterate,
This situation is not considered. It still handles the first <console>
element as another devices, which means that for console[0] with
serial targetType, it calls callback function another time.
It will cause the problem of address conflicts when assigning
spapr-vio address for serial device on pSeries guest.

For pSeries guest, the serial configuration in the xml file
is as the following:
         <serial type='pty'>
               <target port='0'/>
               <address type='spapr-vio'/>
          </serial>

Console configuration is default, the dumped xml file is as the following:
   <serial type='pty'>
      <source path='/dev/pts/5'/>
      <target port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty' tty='/dev/pts/5'>
      <source path='/dev/pts/5'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>

It shows that the <console> device is the alias of serial device.
So its address is the same as the serial device. When detecting
the conflicts in the qemuAssignSpaprVIOAddress the first console
and the serial device conflicts because virDomainDeviceInfoIterate()
still handle these as two different devices, and in the qemuAssignSpaprVIOAddress(),
it will compare these two devices' addressed. If they have same address,
it will report address conflict error.

So this patch is to handle the first console which targetType is serial
as the alias of serial device to avoid address conflicts error reported.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>

This is not that ideal as API for other objects, as it's still
O(n). Because interface driver uses netcf APIs to manage the
stuffs, instead of by itself. And netcf APIs don't return a object.
It provides APIs like old libvirt APIs:

   ncf_number_of_interfaces
   ncf_list_interfaces
   ncf_lookup_by_name
   ......

Perhaps we should further improve netcf to let it provide an API
to return the object, but it could be a later patch. And anyway,
we will still benefit from the new API for the simplification,
and no race like the old APIs.

src/interface/netcf_driver.c: Implement listAllInterfaces

The RPC generator doesn't support returning list of object yet, this patch
do the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchConnectListAllInterfaces.

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllInterfaces.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_INTERFACES and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

This is to list the interface objects, supported filtering flags
are: active|inactive.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllInterfaceFlags
                              and virConnectListAllInterfaces.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllInterfaces)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

Should not return 0 when failed to setup cgroup.

Pushed under trivial rule.

The remote driver first looks at the libvirt auth config file to
fill in any credentials. It then invokes the auth callback for
any remaining credentials. It was accidentally invoking the
auth callback even if there were not any more credentials
required.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

All public API functions must call virResetLastError to clear
out any previous error. The virConnectOpen* functions forgot
to do this.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

src/test/test_driver.c: Implement listAllNetworks.

src/network/bridge_driver.c: Implement listAllNetworks.

src/conf/network_conf.c: Add virNetworkMatch to filter the networks;
and virNetworkList to iterate over all the networks with the filter.

src/conf/network_conf.h: Declare virNetworkList and define the macros
for filters.

src/libvirt_private.syms: Export virNetworkList.

The RPC generator doesn't support returning list of object, this patch
do the work manually.

* daemon/remote.c:
  Implemente the server side handler remoteDispatchConnectListAllNetworks.

* src/remote/remote_driver.c:
  Add remote driver handler remoteConnectListAllNetworks.

* src/remote/remote_protocol.x:
  New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_NETWORKS and
  structs to represent the args and ret for it.

* src/remote_protocol-structs: Likewise.

This is to list the network objects, supported filtering flags
are: active|inactive, persistent|transient, autostart|no-autostart.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllNetworkFlags
                              and virConnectListAllNetworks.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllNetworks)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

e5a1bee0 introduced a regression in Boxes: when Boxes is left idle
(it's still doing some libvirt calls in the background), the
libvirt connection gets closed after a few minutes. What happens is
that this code in virNetClientIOHandleOutput gets triggered:

if (!thecall)
    return -1; /* Shouldn't happen, but you never know... */

and after the changes in e5a1bee0, this causes the libvirt connection
to be closed.

Upon further investigation, what happens is that
virNetClientIOHandleOutput is called from gvir_event_handle_dispatch
in libvirt-glib, which is triggered because the client fd became
writable. However, between the times gvir_event_handle_dispatch
is called, and the time the client lock is grabbed and
virNetClientIOHandleOutput is called, another thread runs and
completes the current call. 'thecall' is then NULL when the first
thread gets to run virNetClientIOHandleOutput.

After describing this situation on IRC, danpb suggested this:

11:37 < danpb> In that case I think the correct thing would be to change
               'return -1' above to 'return 0' since that's not actually an
               error - its a rare, but expected event

which is what this patch is doing. I've tested it against master
libvirt, and I didn't get disconnected in ~10 minutes while this
happens in less than 5 minutes without this patch.

src/test/test_driver.c: Implement poolListAllVolumes.

src/storage/storage_driver.c: Implement poolListAllVolumes.

The RPC generator doesn't returning support list of object, this
patch do the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchStoragePoolListAllVolumes

  * src/remote/remote_driver.c:
    Add remote driver handler remoteStoragePoolListAllVolumes

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_STORAGE_POOL_LIST_ALL_VOLUMES and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

Simply returns the storage volume objects. No supported filter
flags.

include/libvirt/libvirt.h.in: Declare the API
python/generator.py: Skip the function for generating. virStoragePool.py
                     will be added in later patch.
src/driver.h: virDrvStoragePoolListVolumesFlags
src/libvirt.c: Implementation for the API.
src/libvirt_public.syms: Export the symbol to public

GNOME Boxes sometimes stops getting domain events from libvirtd, even
after restarting it. Further investigation in libvirtd shows that
events are properly queued with virDomainEventStateQueue, but the
timer virDomainEventTimer which flushes the events and sends them to
the clients never gets called. Looking at the event queue in gdb
shows that it's non-empty and that its size increases with each new
events.

virDomainEventTimer is set up in virDomainEventStateRegister[ID]
when going from 0 client connecte to 1 client connected, but is
initially disabled. The timer is removed in
virDomainEventStateRegister[ID] when the last client is disconnected
(going from 1 client connected to 0).

This timer (which handles sending the events to the clients) is
enabled in virDomainEventStateQueue when queueing an event on an
empty queue (queue containing 0 events). It's disabled in
virDomainEventStateFlush after flushing the queue (ie removing all
the elements from it). This way, no extra work is done when the queue
is empty, and when the next event comes up, the timer will get
reenabled because the queue will go from 0 event to 1 event, which
triggers enabling the timer.

However, with this Boxes bug, we have a client connected (Boxes), a
non-empty queue (there are events waiting to be sent), but a disabled
timer, so something went wrong.

When Boxes connects (it's the only client connecting to the libvirtd
instance I used for debugging), the event timer is not set as expected
(state->timer == -1 when virDomainEventStateRegisterID is called),
but at the same time the event queue is not empty. In other words,
we had no clients connected, but pending events. This also explains
why the timer never gets enabled as this is only done when an event
is queued on an empty queue.

I think this can happen if an event gets queued using
virDomainEventStateQueue and the client disconnection happens before
the event timer virDomainEventTimer gets a chance to run and flush
the event. In this situation, virDomainEventStateDeregister[ID] will
get called with a non-empty event queue, the timer will be destroyed
if this was the only client connected. Then, when other clients connect
at a later time, they will never get notified about domain events as
the event timer will never get enabled because the timer is only
enabled if the event queue is empty when virDomainEventStateRegister[ID]
gets called, which will is no longer the case.

To avoid this issue, this commit makes sure to remove all events from
the event queue when the last client in unregistered. As there is
no longer anyone interested in receiving these events, these events
are stale so there is no need to keep them around. A client connecting
later will have no interest in getting events that happened before it
got connected.

The introduction of /sys/fs/cgroup came in fairly recent kernels.
Prior to that time distros would pick a custom directory like
/cgroup or /dev/cgroup. We need to auto-detect where this is,
rather than hardcoding it
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add some non-null annotations to qemuMonitorOpen and also
check that the error callback is set, since it is mandatory
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

While PMSUSPENDED state was added a long time ago, we didn't have
corresponding life cycle event.

This patch adds a helper to deal with assigning values to
virTypedParameter structures from strings. The helper parses the value
from the string and assigns it to the corresponding union value.

The quota and period tunables for cpu scheduler accept only a certain
range of values. When changing the live configuration invalid values get
rejected. This check is not performed when changing persistent config.

This patch adds a separate range check, that improves error messages
when changing live config and adds the check for persistent config.
This check is done only when using the API. It is still possible to
specify invalid values in the XML.

This patch tries to clean the code up a little bit and shorten very long
lines.

The apparent semantic change from moving the condition before calling
the setter function is a non-issue here as the setter function is a
no-op when called with both arguments zero.

Commit 2a41bc95 dropped a dependency on gawk, but we can go one step
further and avoid awk altogether.

* src/nwfilter/nwfilter_ebiptables_driver.c
(iptablesLinkIPTablesBaseChain): Simplify command.
(ebiptablesDriverInit, ebiptablesDriverShutdown): Drop awk probe.

This patch removed the "--filterwin2k" dnsmasq command line
parameter which was unnecessary for domain specification,
possibly blocked some usage, and was command line clutter.

Gene Czarcinski <gene@czarc.net>

FreeBSD and OpenBSD have a <net/if.h> that is not self-contained;
and mingw lacks the header altogether.  But gnulib has just taken
care of that for us, so we might as well simplify our code.  In
the process, I got a syntax-check failure if we don't also take
the gnulib execinfo module.

* .gnulib: Update to latest, for execinfo and net_if.
* bootstrap.conf (gnulib_modules): Add execinfo and net_if modules.
* configure.ac: Let gnulib check for headers.  Simplify check for
'struct ifreq', while also including enough prereq headers.
* src/internal.h (IF_NAMESIZE): Drop, now that gnulib guarantees it.
* src/nwfilter/nwfilter_learnipaddr.h: Use correct header for
IF_NAMESIZE.
* src/util/virnetdev.c (includes): Assume <net/if.h> exists.
* src/util/virnetdevbridge.c (includes): Likewise.
* src/util/virnetdevtap.c (includes): Likewise.
* src/util/logging.c (includes): Assume <execinfo.h> exists.
(virLogStackTraceToFd): Handle gnulib's fallback implementation.