This patch adds a couple more nwfilter test cases for the XML parser tests.

Remove symbols that don't exist anymore (e.g. ERROR0) or aren't
message generating functions (e.g. VIR_FREE) or are now reported
as unmarked because the grep command is different, but that should
not be marked at all (e.g. DEBUG0).

Also don't restrict one of the grep lines in the
libvirt_unmarked_diagnostics rule to match exactly one space
between function name and opening parenthesis.

Also unify error/errorf to remoteError and update cfg.mk accordingly.

Fix invalid code generating in esx_vi_generator.py regarding deep copy
types that contain enum properties.

Add strptime and timegm to bootstrap.conf. Both are used to convert a
xsd:dateTime to calendar time.

Add a testcase of the xsd:dateTime conversion.

The MinGW linker needs the libvirt.def file.

The generator was disabled for the new event callbacks, since they
need to be hand written. This patch  adds the C and python glue to
expose the new APIs in the python binding. The python example
program is extended to demonstrate of the code

* python/libvirt-override.c: Registration and dispatch of events
   at the C layer
* python/libvirt-override-virConnect.py: Python glue for events
* examples/domain-events/events-python/event-test.py: Demo use
  of new event callbacks

* po/POTFILES.in: Add src/nwfilter/nwfilter_learnipaddr.c

The network filter / snapshot / hooks code introduced some
non-portable pices that broke the win32 build

* configure.ac: Check for net/ethernet.h required by nwfile config
   parsing code
* src/conf/nwfilter_conf.c: Define ethernet protocol  constants
  if net/ethernet.h is missing
* src/util/hooks.c: Disable hooks build on Win32 since it lacks
  fork/exec/pipe
* src/util/threads-win32.c: Fix unchecked return value
* tools/virsh.c: Disable SIGPIPE on Win32 since it doesn't exist.
  Fix non-portable strftime() formats

Changes from V1 to V2 of this patch
- I had reversed the logic thinking that icmp type 0 is a echo
request,but it's reply -- needed to reverse the logic
- Found that ebtables takes the --ip-tos argument only as a hex number

This patch enables the skipping of some of the ICMP traffic rules on the
iptables level under certain circumstances so that the following filter
properly enables unidirectional pings:

<filter name='testcase'>
    <uuid>d6b1a2af-def6-2898-9f8d-4a74e3c39558</uuid>
    <!-- allow incoming ICMP Echo Request -->
    <rule action='accept' direction='in' priority='500'>
        <icmp type='8'/>
    </rule>
    <!-- allow outgoing ICMP Echo Reply -->
    <rule action='accept' direction='out' priority='500'>
        <icmp type='0'/>
    </rule>
    <!-- drop all other ICMP traffic -->
    <rule action='drop' direction='inout' priority='600'>
        <icmp/>
    </rule>
</filter>

Extend tests to cover all SCSI controller types and document the
new type.

The lsisas1068 SCSI controller type was added in ESX 4.0. The VMX
parser reports an error when this controller type is present. This
makes virsh dumpxml fail for every domain that uses this controller
type.

This patch fixes this and adds lsisas1068 to the list of accepted
SCSI controller types.

Reported by Jonathan Kelley.

Just checking for a windres tool might hit even on Linux systems when
building for Linux (e.g.: when using Gentoo and having built binutils
with multitarget support), and will then fail to link properly at the
end of the build.

* configure.ac: Avoid searching for windres on non windows target

Removing the tests/nwfilterxml2xmltest executable that got in with a previous patch.

This patch implements support for learning a VM's IP address. It uses
the pcap library to listen on the VM's backend network interface (tap)
or the physical ethernet device (macvtap) and tries to capture packets
with source or destination MAC address of the VM and learn from DHCP
Offers, ARP traffic, or first-sent IPv4 packet what the IP address of
the VM's interface is. This then allows to instantiate the network
traffic filtering rules without the user having to provide the IP
parameter somewhere in the filter description or in the interface
description as a parameter. This only supports to detect the parameter
IP, which is for the assumed single IPv4 address of a VM. There is not
support for interfaces that may have multiple  IP addresses (IP
aliasing) or IPv6 that may then require more than one valid IP address
to be detected. A VM can have multiple independent interfaces that each
uses a different IP address and in that case it will be attempted to
detect each one of the address independently.

So, when for example an interface description in the domain XML has
looked like this up to now:

    <interface type='bridge'>
      <source bridge='mybridge'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'>
        <parameter name='IP' value='10.2.3.4'/>
      </filterref>
    </interface>

you may omit the IP parameter:

    <interface type='bridge'>
      <source bridge='mybridge'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'/>
    </interface>

Internally I am walking the 'tree' of a VM's referenced network filters
and determine with the given variables which variables are missing. Now,
the above IP parameter may be missing and this causes a libvirt-internal
thread to be started that uses the pcap library's API to listen to the
backend interface  (in case of macvtap to the physical interface) in an
attempt to determine the missing IP parameter. If the backend interface
disappears the thread terminates assuming the VM was brought down. In
case of a macvtap device a timeout is being used to wait for packets
from the given VM (filtering by VM's interface MAC address). If the VM's
macvtap device disappeared the thread also terminates. In all other
cases it tries to determine the IP address of the VM and will then apply
the rules late on the given interface, which would have happened
immediately if the IP parameter had been explicitly given. In case an
error happens while the firewall rules are applied, the VM's backend
interface is 'down'ed preventing it to communicate. Reasons for failure
for applying the network firewall rules may that an ebtables/iptables
command failes or OOM errors. Essentially the same failure reasons may
occur as when the firewall rules are applied immediately on VM start,
except that due to the late application of the filtering rules the VM
now is already running and cannot be hindered anymore from starting.
Bringing down the whole VM would probably be considered too drastic.
While a VM's IP address is attempted to be determined only limited
updates to network filters are allowed. In particular it is prevented
that filters are modified in such a way that they would introduce new
variables.

A caveat: The algorithm does not know which one is the appropriate IP
address of a VM. If the VM spoofs an IP address in its first ARP traffic
or IPv4 packets its filtering rules will be instantiated for this IP
address, thus 'locking' it to the found IP address. So, it's still
'safer' to explicitly provide the IP address of a VM's interface in the
filter description if it is known beforehand.

* configure.ac: detect libpcap
* libvirt.spec.in: require libpcap[-devel] if qemu is built
* src/internal.h: add the new ATTRIBUTE_PACKED define
* src/Makefile.am src/libvirt_private.syms: add the new modules and symbols
* src/nwfilter/nwfilter_learnipaddr.[ch]: new module being added
* src/nwfilter/nwfilter_driver.c src/conf/nwfilter_conf.[ch]
  src/nwfilter/nwfilter_ebiptables_driver.[ch]
  src/nwfilter/nwfilter_gentech_driver.[ch]: plu the new functionality in
* tests/nwfilterxml2xmltest: extend testing

* src/xen/xm_internal.c (xenXMDomainDefineXML): Remove useless and
leak-inducing call to virGetDomain, as well as decl of now-unused local.

* src/storage/storage_backend.c (createRawFileOpHook): Remove dead
stores and declaration of each stored-to variable.

* src/qemu/qemu_driver.c (qemudDomainGetSecurityLabel): Remove store
and declaration.

When comparing a CPU to host CPU, the result would be
VIR_CPU_COMPARE_SUPERSET (or even VIR_CPU_COMPARE_INCOMPATIBLE if strict
match was required) even though the two CPUs were identical.

No change in semantics.

There's no sense in advertising cpuselection capability when host CPU
is not properly detected and advertised in host capabilities.

When qemu libvirt driver doesn't support guest CPU selection with given
qemu binary, guests requiring specific CPU should fail to start instead
of being silently supplied with a default CPU.

* src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Remove dead store
into "snap", as well as its declaration.

git grep found 12 of the former but 100 of the latter in src/.

* src/remote/remote_driver.c (initialise_gnutls): Rename...
(initialize_gnutls): ...to this.
(doRemoteOpen): Adjust caller.
* src/xen/xen_driver.c (xenUnifiedOpen): Adjust output string.
* src/util/network.c: Adjust comments.
Suggested by Matthias Bolte.

* src/conf/domain_event.c (virDomainEventGraphicsNewFromDom):
Return NULL when handling out-of-memory error, rather than
falling through with ev=NULL and then assigning to ev->member.
(virDomainEventGraphicsNewFromObj): Likewise.

The attached patch fixes a problem due to the mac match in iptables only
supporting --mac-source and no --mac-destination, thus it not being
symmetric. Therefore a rule like this one

<rule action='drop' direction='out'>
  <all match='no' srcmacaddr='$MAC'/>
</rule>

should only have the MAC match on traffic leaving the VM and not test
for the same source MAC address on traffic that the VM receives.

* src/qemu/qemu_driver.c (qemudStartVMDaemon): Initialize "logfile"
to ensure that we don't use it uninitialized -- thus closing an
arbitrary file descriptor -- in the cleanup block.

* tests/virt-aa-helper-test: test augmented with hostdev and sdl display
  checks

* examples/apparmor/libvirt-qemu examples/apparmor/usr.sbin.libvirtd
  examples/apparmor/usr.lib.libvirt.virt-aa-helper: Update the examples

* src/security/virt-aa-helper.c: add support for SDL devices and 3
  code cleanups

* src/security/virt-aa-helper.c: adjust virt-aa-helper to handle pci
  devices. Update valid_path() to have an override array to check against,
  and add "/sys/devices/pci" to it. Then rename file_iterate_cb() to
  file_iterate_hostdev_cb() and create file_iterate_pci_cb() based on it

adjust virt-aa-helper to handle backing store
* src/security/virt-aa-helper.c: look for backing store metadata
  for disk definitions.

To avoid an error when hitting the <seclabel...> definition
* src/security/virt-aa-helper.c: add VIR_DOMAIN_XML_INACTIVE flag
  to virDomainDefParseString

Don't exit with error if the user unloaded the profile outside of
 libvirt
* src/security/virt-aa-helper.c: check the exit error from apparmor_parser
  before exiting with a failure

The calls to virExec() in security_apparmor.c when
invoking virt-aa-helper use VIR_EXEC_CLEAR_CAPS. When compiled without
libcap-ng, this is not a problem (it's effectively a no-op) but with
libcap-ng this causes MAC_ADMIN to be cleared. MAC_ADMIN is needed by
virt-aa-helper to manipulate apparmor profiles and without it VMs will
not start[1]. This patch calls virExec with the default VIR_EXEC_NONE
instead.
* src/security/security_apparmor.c: fallback to VIR_EXEC_NONE flags for
  virExec of virt_aa_helper

Reported by Paul Jenner