Silently ignored flags get in the way of new features that
use those flags.  Also, an upcoming syntax check will favor
unsigned flags.

* src/nodeinfo.h (nodeGetCPUStats, nodeGetMemoryStats): Drop
unused attribute.
* src/interface/netcf_driver.c (interfaceOpenInterface)
(interfaceDefineXML, interfaceCreate, interfaceDestroy): Reject
unknown flags.
* src/network/bridge_driver.c (networkOpenNetwork)
(networkGetXMLDesc): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterOpen): Likewise.
* src/secret/secret_driver.c (secretOpen, secretDefineXML)
(secretGetXMLDesc, secretSetValue): Likewise.
* src/util/logging.c (virLogDefineFilter, virLogDefineOutput)
(virLogMessage): Likewise; also use unsigned flags.
* src/util/logging.h (virLogDefineFilter, virLogDefineOutput)
(virLogMessage): Change signature.
* src/util/command.c (virExecWithHook): Likewise.

Now that the public APIs always use unsigned flags, the internal
driver callbacks might as well do likewise.

* src/driver.h (vrDrvOpen, virDrvDomainCoreDump)
(virDrvDomainGetXMLDesc, virDrvNetworkGetXMLDesc)
(virDrvNWFilterGetXMLDesc): Update type.
* src/remote/remote_protocol.x (remote_open_args)
(remote_domain_core_dump_args, remote_domain_get_xml_desc_args)
(remote_network_get_xml_desc_args)
(remote_nwfilter_get_xml_desc_args): Likewise.
* src/test/test_driver.c: Update clients.
* src/remote/remote_driver.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
* src/xen/xen_hypervisor.h: Likewise.
* src/xen/xen_driver.c: Likewise.
* src/xen/xend_internal.c: Likewise.
* src/xen/xend_internal.h: Likewise.
* src/xen/xm_internal.c: Likewise.
* src/xen/xm_internal.h: Likewise.
* src/xen/xs_internal.c: Likewise.
* src/xen/xs_internal.h: Likewise.
* src/xen/xen_inotify.c: Likewise.
* src/xen/xen_inotify.h: Likewise.
* src/phyp/phyp_driver.c: Likewise.
* src/openvz/openvz_driver.c: Likewise.
* src/vmware/vmware_driver.c: Likewise.
* src/vbox/vbox_driver.c: Likewise.
* src/vbox/vbox_tmpl.c: Likewise.
* src/xenapi/xenapi_driver.c: Likewise.
* src/esx/esx_driver.c: Likewise.
* src/esx/esx_interface_driver.c: Likewise.
* src/esx/esx_network_driver.c: Likewise.
* src/esx/esx_storage_driver.c: Likewise.
* src/esx/esx_device_monitor.c: Likewise.
* src/esx/esx_secret_driver.c: Likewise.
* src/esx/esx_nwfilter_driver.c: Likewise.
* src/interface/netcf_driver.c: Likewise.
* src/nwfilter/nwfilter_driver.c: Likewise.
* src/libxl/libxl_driver.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/lxc/lxc_driver.c: Likewise.
* src/uml/uml_driver.c: Likewise.
* src/network/bridge_driver.c: Likewise.
* src/secret/secret_driver.c: Likewise.
* src/storage/storage_driver.c: Likewise.
* src/node_device/node_device_hal.c: Likewise.
* src/node_device/node_device_udev.c: Likewise.
* src/remote_protocol-structs: Likewise.

Some callers expected virFileMakePath to set errno, some expected
it to return an errno value. Unify this to return 0 on success and
-1 on error. Set errno to report detailed error information.

Also optimize virFileMakePath if stat fails with an errno different
from ENOENT.

networkSaveDnsmasqHostsfile was added in 8fa9c221 (Apr 2010).
It has a force flag. If the dnsmasq hostsfile already exists force
needs to be true to overwrite it. networkBuildDnsmasqArgv sets force
to false, networkDefine sets it to true. This results in the
hostsfile being written only in networkDefine in the common case.
If no error occurred networkSaveDnsmasqHostsfile returns true and
networkBuildDnsmasqArgv adds the --dhcp-hostsfile to the dnsmasq
command line.

networkSaveDnsmasqHostsfile was changed in 89ae9849 (24 Jun 2011)
to return a new dnsmasqContext instead of reusing one. This change broke
the logic of the force flag as now networkSaveDnsmasqHostsfile returns
NULL on error, but the early return -- if force was not set and the
hostsfile exists -- returns 0. This turned the early return in an error
case and networkBuildDnsmasqArgv didn't add the --dhcp-hostsfile option
anymore if the hostsfile already exists. It did because networkDefine
created the hostsfile already.

Then 9d4e2845 fixed the return 0 case in networkSaveDnsmasqHostsfile
but didn't apply the force option correctly to the new addnhosts file.
Now force doesn't control an early return anymore, but influences the
handling of the hostsfile context creation and dnsmasqSave is always
called now. This commit also added test cases that reveal several
problems. First, the tests now calls functions that try to write the
dnsmasq config files to disk. If someone runs this tests as root this
might overwrite actively used dnsmasq config files, this is a no-go. Also
the tests depend on configure --localstatedir, this needs to be fixed as
well, because it makes the tests fail when localstatedir is different
from /var.

This patch does several things to fix this:

1) Move dnsmasqContext creation and saving out of networkBuildDnsmasqArgv
to the caller to separate the command line generation from the config
file writing. This makes the command line generation testable without the
risk of interfering with system files, because the tests just don't call
dnsmasqSave.

2) This refactoring of networkSaveDnsmasqHostsfile makes the force flag
useless as the saving happens somewhere else now. This fixes the wrong
usage of the force flag in combination with then newly added addnhosts
file by removing the force flag.

3) Adapt the wrong test cases to the correct behavior, by adding the
missing --dhcp-hostsfile option. Both affected tests contain DHCP host
elements but missed the necessary --dhcp-hostsfile option.

4) Rename networkSaveDnsmasqHostsfile to networkBuildDnsmasqHostsfile,
because it doesn't save the dnsmasqContext anymore.

5) Move all directory creations in dnsmasq context handling code from
the *New functions to dnsmasqSave to avoid directory creations in system
paths in the test cases.

6) Now that networkBuildDnsmasqArgv doesn't create the dnsmasqContext
anymore the test case can create one with the localstatedir that is
expected by the tests instead of the configure --localstatedir given one.

If a domain name is defined for a network, add the --expand-hosts
option to the dnsmasq commandline. This results in the domain being
added to any hostname that is defined in a dns <host> element and
contains no '.' characters (i.e. it is an "unqualified"
hostname). Since PTR records are automatically created for any name
defined in <host>, the result of a PTR request will change from the
unqualified name to the qualified name.

This also has the same effect on any hostnames that dnsmasq reads
from the host's /etc/hosts file.

(In the case of guest hostnames that were learned by dnsmasq via DHCP
requests, they were already getting the domain name added on, even
without --expand-hosts).

Convert networkDnsmasqLeaseFileName to a replaceable function pointer
that allow the testsuite to use a version of that function that is not
depending on configure --localstatedir.

This fixes 5 of 6 test failures, when configure --localstatedir isn't
set to /var.

This block was inadvertently added with the wrong indentation.

This commit introduces names definition for the DNS hosts file using
the following syntax:

  <dns>
    <host ip="192.168.1.1">
      <name>alias1</name>
      <name>alias2</name>
    </host>
  </dns>

Some of the improvements and fixes were done by Laine Stump so
I'm putting him into the SOB clause again ;-)
Signed-off-by: NMichal Novotny <minovotn@redhat.com>
Signed-off-by: NLaine Stump <laine@laine.org>

Signed-off-by: NMichal Novotny <minovotn@redhat.com>

The dnsmasq commandline was being built as a part of running
dnsmasq. This patch puts the commandline build into a separate
function (and exports it as a private API) making it possible to build
a dnsmasq commandline without executing it, so that we can write a
test program to verify that the proper commandlines are being created.
Signed-off-by: NMichal Novotny <minovotn@redhat.com>

This commit introduces the <dns> element and <txt> record for the
virtual DNS network. The DNS TXT record can be defined using following
syntax in the network XML file:

  <dns>
    <txt name="example" value="example value" />
  </dns>

Also, the Relax-NG scheme has been altered to allow the texts without
spaces only for the name element and some nitpicks about memory
free'ing have been fixed by Laine so therefore I'm adding Laine to the
SOB clause ;-)
Signed-off-by: NMichal Novotny <minovotn@redhat.com>
Signed-off-by: NLaine Stump <laine@laine.org>

Since we virEventRegisterDefaultImpl is now a public API, callers need
a way to invoke the default registered Handle and Timeout functions. We
already have general functions for these internally, so promote
them to the public API.

v2:
    Actually add APIs to libvirt.h

Add many version number annotations to the internal driver
tables, to allow hvsupport.html to display more accurate
information

Change all the driver struct initializers to use the
C99 style, leaving out unused fields. This will make
it possible to add new APIs without changing every
driver. eg change:

    qemudDomainResume, /* domainResume */
    qemudDomainShutdown, /* domainShutdown */
    NULL, /* domainReboot */
    qemudDomainDestroy, /* domainDestroy */

to

    .domainResume = qemudDomainResume,
    .domainShutdown = qemudDomainShutdown,
    .domainDestroy = qemudDomainDestroy,

And get rid of any existing C99 style initializersr which
set NULL, eg change

     .listPools          = vboxStorageListPools,
     .numOfDefinedPools  = NULL,
     .listDefinedPools   = NULL,
     .findPoolSources    = NULL,
     .poolLookupByName   = vboxStoragePoolLookupByName,

to

     .listPools          = vboxStorageListPools,
     .poolLookupByName   = vboxStoragePoolLookupByName,

We were 31/73 on whether to translate; since less than 50% translated
and since VIR_INFO is less than VIR_WARN which also doesn't translate,
this makes sense.

* cfg.mk (sc_prohibit_gettext_markup): Add VIR_INFO, since it
falls between WARN and DEBUG.
* daemon/libvirtd.c (qemudDispatchSignalEvent, remoteCheckAccess)
(qemudDispatchServer): Adjust offenders.
* daemon/remote.c (remoteDispatchAuthPolkit): Likewise.
* src/network/bridge_driver.c (networkReloadIptablesRules)
(networkStartNetworkDaemon, networkShutdownNetworkDaemon)
(networkCreate, networkDefine, networkUndefine): Likewise.
* src/qemu/qemu_driver.c (qemudDomainDefine)
(qemudDomainUndefine): Likewise.
* src/storage/storage_driver.c (storagePoolCreate)
(storagePoolDefine, storagePoolUndefine, storagePoolStart)
(storagePoolDestroy, storagePoolDelete, storageVolumeCreateXML)
(storageVolumeCreateXMLFrom, storageVolumeDelete): Likewise.
* src/util/bridge.c (brProbeVnetHdr): Likewise.
* po/POTFILES.in: Drop src/util/bridge.c.

These VIR_XXXX0 APIs make us confused, use the non-0-suffix APIs instead.

How do these coversions works? The magic is using the gcc extension of ##.
When __VA_ARGS__ is empty, "##" will swallow the "," in "fmt," to
avoid compile error.

example: origin				after CPP
	high_level_api("%d", a_int)	low_level_api("%d", a_int)
	high_level_api("a  string")	low_level_api("a  string")

About 400 conversions.

8 special conversions:
VIR_XXXX0("") -> VIR_XXXX("msg") (avoid empty format) 2 conversions
VIR_XXXX0(string_literal_with_%) -> VIR_XXXX(%->%%) 0 conversions
VIR_XXXX0(non_string_literal) -> VIR_XXXX("%s", non_string_literal)
  (for security) 6 conversions
Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>

This matches the public API and helps to get rid of some special
case code in the remote generator.

Rename driver API functions and XDR protocol structs.

No functional change included outside of the remote generator.

We already have virAsprintf, so picking a similar name helps for
seeing a similar purpose.  Furthermore, the prefix V before printf
generally implies 'va_list', even though this variant was '...', and
the old name got in the way of adding a new va_list version.

global rename performed with:

$ git grep -l virBufferVSprintf \
  | xargs -L1 sed -i 's/virBufferVSprintf/virBufferAsprintf/g'

then revert the changes in ChangeLog-old.

otherwise the directory returned by networkDnsmasqLeaseFileName will not
be created if ipdef->nhosts == 0 in networkBuildDnsmasqArgv.

And from all related macros and functions.

This patch addresses:

   https://bugzilla.redhat.com/show_bug.cgi?id=694382

In order to give each libvirt-created bridge a fixed MAC address,
commit 5754dbd5, added code to create
a dummy tap device with guaranteed lowest MAC address and attach it to
the bridge. This tap device was given the name "${bridgename}-nic".
However, an interface device name must be IFNAMSIZ (15) characters or
less, so a valid ${bridgename} such as "verylongname123" (15
characters) would lead to an invalid tap device name
("verylongname123-nic" - 19 characters), and that in turn led to a
failure to bring up the network.

The solution is to shorten the part of the original name used to
generate the tap device name. However, simply truncating it is
insufficient, because the last few characters of an interface name are
often a number used to indicate one of a list of several similar
devices (for example, "verylongname123", "verylongname124", etc) and
simple truncation would lead to duplicate names (eg "verlongnam-nic"
and "verylongnam-nic"). So instead we take the first 8 characters of
$bridgename ("verylong" in the example), add on the final 3 bytes
("123"), then add "-nic" (so "verylong123-nic").  Not pretty, but it
is much more likely to generate a unique name, and is reproducible
(unlike, say, a random number).

This fixes: https://bugzilla.redhat.com/show_bug.cgi?id=696660

While starting a network, if brSetForwardDelay() fails, we go to err1
where we want to access macTapIfName variable which was just
VIR_FREE'd a few lines above. Instead, keep macTapIfName until we are
certain of success.

Currently libvirt's default logging is limited and it is difficult to
determine what was happening when a proglem occurred (especially on a
machines where one don't know the detail.)  This patch helps to do that
by making additional logging available for the following events:

  creating/defining/undefining domains
  creating/defining/undefining/starting/stopping networks
  creating/defining/undefining/starting/stopping storage pools
  creating/defining/undefining/starting/stopping storage volumes.

* AUTHORS: add Naoya Horiguchi
* src/network/bridge_driver.c src/qemu/qemu_driver.c
  src/storage/storage_driver.c: provide more VIR_INFO logging

This simplifies several callers that were repeating checks already
guaranteed by util.c, and makes other callers more robust to now
reject directories.  remote_driver.c was over-strict - access(,R_OK)
is only needed to execute a script file; a binary only needs
access(,X_OK) (besides, it's unusual to see a file with x but not
r permissions, whether script or binary).

* cfg.mk (sc_prohibit_access_xok): New syntax-check rule.
(exclude_file_name_regexp--sc_prohibit_access_xok): Exempt one use.
* src/network/bridge_driver.c (networkStartRadvd): Fix offenders.
* src/qemu/qemu_capabilities.c (qemuCapsProbeMachineTypes)
(qemuCapsInitGuest, qemuCapsInit, qemuCapsExtractVersionInfo):
Likewise.
* src/remote/remote_driver.c (remoteFindDaemonPath): Likewise.
* src/uml/uml_driver.c (umlStartVMDaemon): Likewise.
* src/util/hooks.c (virHookCheck): Likewise.

This is detailed in:

  https://bugzilla.redhat.com/show_bug.cgi?id=688957

Since radvd is executed by daemonizing it, the attempt to exec the
radvd binary doesn't happen until after libvirtd has already received
an exit code from the intermediate forked process, so no error is
detected or logged by __virExec().

We can't require radvd as a prerequisite for the libvirt package (many
installations don't use IPv6, so they don't need it), so instead we
add in a check to verify there is an executable radvd binary prior to
trying to exec it.

Normally dnsmasq will send a default route (the address of the host in
the network definition) to any client requesting an address via
DHCP. On an isolated network this makes no sense, as we have iptables
to prevent any traffic going out via that interface, so anything sent
that way would be dropped anyway.

This extra/unusable default route becomes problematic if you have
setup a guest with multiple network interfaces, with one connected to
an isolated network and another that provides connectivity to the
outside (example - one interface directly connecting to a physical
interface via macvtap, with a second connected to an isolated network
so that the host and guest can communicate (macvtap doesn't support
guest<->host communication without an external switch that supports
vepa, or reflecting all traffic back)). In this case, if the guest
chooses the default route of the isolated network, the guest will not
be able to get network traffic beyond the host.

To prevent dnsmasq from sending a default route, you can tell it to
send 0 bytes of data for the default route option (option number 3)
with --dhcp-option=3 (normally the data to send for the option would
follow the option number; no extra data means "don't send this option").

I have checked on RHEL5 (a good representative of the oldest supported
libvirt platforms) and its version of dnsmasq (2.45) does support
--dhcp-option, so this shouldn't create any compatibility problems.

By default, all dnsmasq processes share the same leases file. libvirt
also uses the --dhcp-lease-max option to control the maximum number of
leases allowed. The problem is that libvirt puts in a number equal to
the number of addresses in the range for the one network handled by a
single instance of dnsmasq, but dnsmasq checks the total number of
leases in the file (which could potentially contain many more).

The solution is to tell each instance of dnsmasq to create and use its
own leases file. (/var/lib/libvirt/network/<net-name>.leases).

This file is created by dnsmasq when it starts, but not deleted when
it exists. This is fine when the network is just being stopped, but if
the leases file was left around when a network was undefined, we could
end up with an ever-increasing number of dead files - instead, we
explicitly unlink the leases file when a network is undefined.

Note that Ubuntu carries a patch against an older version of libvirt for this:

hhttps://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/713071
ttp://bazaar.launchpad.net/~serge-hallyn/ubuntu/maverick/libvirt/bugall/revision/109

I was certain I'd also seen discussion of this on libvir-list or
libvirt-users, but couldn't find it.

The previous commit put a large portion of networkBuildDnsmasqArgv
inside an if { } block. This readjusts the indentation.

This fixes a regression introduced in commit ad48df, and reported on
the libvirt-users list:

  https://www.redhat.com/archives/libvirt-users/2011-March/msg00018.html

The problem in that commit was that we began searching a list of ip
address definitions (rather than just having one) to look for a dhcp
range or static host; when we didn't find any, our pointer (ipdef) was
left at NULL, and when ipdef was NULL, we returned without starting up
dnsmasq.

Previously dnsmasq was started even without any dhcp ranges or static
entries, because it's still useful for DNS services.

Another problem I noticed while investigating was that, if there are
IPv6 addresses, but no IPv4 addresses of any kind, we would jump out
at an ever higher level in the call chain.

This patch does the following:

1) networkBuildDnsmasqArgv() = all uses of ipdef are protected from
   NULL dereference. (this patch doesn't change indentation, to make
   review easier. The next patch will change just the
   indentation). ipdef is intended to point to the first IPv4 address
   with DHCP info (or the first IPv4 address if none of them have any
   dhcp info).

2) networkStartDhcpDaemon() = if the loop looking for an ipdef with
   DHCP info comes up empty, we then grab the first IPv4 def from the
   list. Also, instead of returning if there are no IPv4 defs, we just
   return if there are no IP defs at all (either v4 or v6). This way a
   network that is IPv6-only will still get dnsmasq listening for DNS
   queries.

3) in networkStartNetworkDaemon() - we will startup dhcp not just if there
   are any IPv4 addresses, but also if there are any IPv6 addresses.

since dnsmasq >= 2.56 now bails out with empty arguments. See
    http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=613944
for the Debian bug and
    http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589885
for the upstream reasoning.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=609463

The problem was that, since a bridge always acquires the MAC address
of the connected interface with the numerically lowest MAC, as guests
are started and stopped, it was possible for the MAC address to change
over time, and this change in the network was being detected by
Windows 7 (it sees the MAC of the default route change), so on each
reboot it would bring up a dialog box asking about this "new network".

The solution is to create a dummy tap interface with a MAC guaranteed
to be lower than any guest interface's MAC, and attach that tap to the
bridge as soon as it's created. Since all guest MAC addresses start
with 0xFE, we can just generate a MAC with the standard "0x52, 0x54,
0" prefix, and it's guaranteed to always win (physical interfaces are
never connected to these bridges, so we don't need to worry about
competing numerically with them).

Note that the dummy tap is never set to IFF_UP state - that's not
necessary in order for the bridge to take its MAC, and not setting it
to UP eliminates the clutter of having an (eg) "virbr0-nic" displayed
in the output of the ifconfig command.

I chose to not auto-generate the MAC address in the network XML
parser, as there are likely to be consumers of that API that don't
need or want to have a MAC address associated with the
bridge.

Instead, in bridge_driver.c when the network is being defined, if
there is no MAC, one is generated. To account for virtual network
configs that already exist when upgrading from an older version of
libvirt, I've added a %post script to the specfile that searches for
all network definitions in both the config directory
(/etc/libvirt/qemu/networks) and the state directory
(/var/lib/libvirt/network) that are missing a mac address, generates a
random address, and adds it to the config (and a matching address to
the state file, if there is one).

docs/formatnetwork.html.in: document <mac address.../>
docs/schemas/network.rng: add nac address to schema
libvirt.spec.in: %post script to update existing networks
src/conf/network_conf.[ch]: parse and format <mac address.../>
src/libvirt_private.syms: export a couple private symbols we need
src/network/bridge_driver.c:
    auto-generate mac address when needed,
    create dummy interface if mac address is present.
tests/networkxml2xmlin/isolated-network.xml
tests/networkxml2xmlin/routed-network.xml
tests/networkxml2xmlout/isolated-network.xml
tests/networkxml2xmlout/routed-network.xml: add mac address to some tests

* src/network/bridge_driver.c (networkStartNetworkDaemon): Delete
unused assignments.

* dnsmasq listens on all defined IPv[46] addresses for network
* Add ip6tables rules to allow DNS traffic to host

VIR_ERR_OPERATION_INVALID means that the operation is not valid
for the current state of the involved object.

VIR_ERR_INVALID_* is meant for invalid pointers only.

I added a host definition to a network definition:

<network>
  <name>Lokal</name>
  <uuid>2074f379-b82c-423f-9ada-305d8088daaa</uuid>
  <bridge name='virbr1' stp='on' delay='0' />
  <ip address='192.168.180.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.180.128' end='192.168.180.254' />
      <host mac='23:74:00:03:42:02' name='somevm' ip='192.168.180.10' />
    </dhcp>
  </ip>
</network>

But due to the wrong if-statement the argument --dhcp-hostsfile doesn't get
added to the dnsmasq command. The patch below fixes it for me.

Running an instance of the router advertisement daemon (radvd) allows
guests using the virtual network to automatically acquire an IPv6
address and default route. Note that acquiring an address only works
for networks with a prefix length of exactly 64 - radvd is still run
in other circumstances, and still advertises routes, but autoconf will
not work because it requires exactly 64 bits of address info from the
network prefix.

This patch avoids a race condition with the pidfile by manually
daemonizing radvd rather than allowing it to daemonize itself, then
creating our own pidfile (in addition to radvd's own file, which is
unnecessary, but there is no way to tell radvd to not create it). This
is accomplished by exec'ing it with "--debug 1" in the commandline,
and using virCommand's features to fork, create a pidfile, and detach
from the newly forked process.

At this point everything is already in place to make IPv6 happen, we just
need to add a few rules, remove some checks for IPv4-only, and document
the changes to the XML on the website.

All of the iptables functions eventually call down to a single
bottom-level function, and fortunately, ip6tables syntax (for all the
args that we use) is identical to iptables format (except the
addresses), so all we need to do is:

1) Get an address family down to the lowest level function in each
   case, either implied through an address, or explicitly when no
   address is in the parameter list, and

2) At the lowest level, just decide whether to call "iptables" or
   "ip6tables" based on the family.

The location of the ip6tables binary is determined at build time by
autoconf. If a particular target system happens to not have ip6tables
installed, any attempts to run it will generate an error, but that
won't happen unless someone tries to define an IPv6 address for a
network. This is identical behavior to IPv4 addresses and iptables.