https://bugzilla.redhat.com/show_bug.cgi?id=860004

Upstream kernel introduced new sysfs knob "merge_across_nodes" to
specify if pages from different numa nodes can be merged. When set
to 0, only pages which physically reside in the memory area of
same NUMA node can be merged. When set to 1, pages from all nodes
can be merged.

This patch supports the tuning by adding new param field
"shm_merge_across_nodes".

SIGINT hasn't been blocked, which could lead to losing it somewhere in
virDomainGetBlockJobInfo and not aborting the job.

Using VIR_DOMAIN_XML_MIGRATABLE flag, one can request domain's XML
configuration that is suitable for migration or save/restore. Such XML
may contain extra run-time stuff internal to libvirt and some default
configuration may be removed for better compatibility of the XML with
older libvirt releases.

This flag may serve as an easy way to get the XML that can be passed
(after desired modifications) to APIs that accept custom XMLs, such as
virDomainMigrate{,ToURI}2 or virDomainSaveFlags.

Add a read-only udev based backend for virInterface. Useful for distros
that do not have netcf support yet. Multiple libvirt based utilities use
a HAL based fallback when virInterface is not available which is less
than ideal. This implements:
* virConnectNumOfInterfaces()
* virConnectListInterfaces()
* virConnectNumOfDefinedInterfaces()
* virConnectListDefinedInterfaces()
* virConnectListAllInterfaces()
* virConnectInterfaceLookupByName()
* virConnectInterfaceLookupByMACString()

I was using qemu-monitor-command during development, and found it quite
hard to use.  Compare the results of this patch on ease of reading:

$ virsh qemu-monitor-command          dom '{"execute":"query-version"}'
{"return":{"qemu":{"micro":1,"minor":12,"major":0},"package":"(qemu-kvm-0.12.1.2)"},"id":"libvirt-7683"}

$ virsh qemu-monitor-command --pretty dom '{"execute":"query-version"}'
{
    "return": {
        "qemu": {
            "micro": 1,
            "minor": 12,
            "major": 0
        },
        "package": "(qemu-kvm-0.12.1.2)"
    },
    "id": "libvirt-7674"
}

* tools/virsh-host.c (cmdQemuMonitorCommand): New option.
* tools/virsh.pod (qemu-monitor-command): Document it.

Clarify that domains with numeric names can only be identified by
their domain id.

This command uses the new virNetworkUpdate() API to modify an existing
network definition, and optionally have those modifications take
effect immediately without restarting the network.

An example usage:

  virsh net-update mynet add-last ip-dhcp-host \
   "<host mac='00:11:22:33:44:55' ip='192.168.122.45'/>" \
   --live --config

If you like, you can instead put the xml into a file, and call like
this:

  virsh net-update mynet add ip-dhcp-host /tmp/myxml.xml
   --live --config

virsh will autodetect whether the argument is itself an xml element,
or if it's a file, by looking at the first character - the first
character of an xml element is always "<", and the first character of
a file is almost always *not* "<" (in the rare case that it is, the
user could specify "./<filename...").

A --parent-index option is also available (to give the index within a
list of parent objects, e.g. the index of the parent <ip> element when
updating ip-dhcp-host elements), but is optional and at least for now
will probably be used rarely.

--live, --config, and --current options - if you specify --live, only
the live state of the network will be updated. If you also specify
--config, then the persistent configuration will also be updated;
these two commands can be given separately, or both together. If you
don't specify either (you can optionally specify "--current" for the
same effect), then the "current" config will be updated (i.e. if the
network is active, then only its live config is affected, but if the
network is inactive, only the persistent config is affected).

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Move the 'Daemon' entry to the Miscellaneous section since its not a
networking driver or component.

Based exclusively on work by Eric Blake in a patch posted with the same
subject. However some modifications related to comments and my plans to
add another backend.

Added WITH_INTERFACE as the only automake variable deciding whether to
build the driver and using WITH_NETCF to identify that we're wanting to
use the netcf library as the backend.

* configure.ac: Added with_interface
* src/interface/netcf_driver.c: Renamed..
* src/interface/interface_backend_netcf.c: ..to this to match storage.
* src/interface/netcf_driver.h: Renamed..
* src/interface/interface_driver.h: ..to this.
* daemon/Makefile.am: Respect WITH_INTERFACE and WITH_NETCF.
* libvirt.spec.in: Add RPM support for --with-interface

Match upstream's naming of QEMU. There was a notice on the ML that said
the preferred spelling was QEMU a while back.

The cleanup path in virsh-edit helper was never reached when the edit
was successful leaking the document in memory as well as the temporary
file.

The new command 'virsh blockcommit $dom $disk' requests the start
of an asynchronous commit operation across the entire chain of
$disk.  Further arguments can fine-tune which portion of the
chain is committed.  Existing 'virsh blockjob' commands can then
track the status, change the bandwidth, or abort the commit job.

With a bit more on the command line, 'virsh blockcommit $dom $disk
--wait --verbose' can be used for blocking behavior, with visual
feedback on the overall status, and can be canceled with Ctrl-C.

The overall design, including the wait loop logic, borrows heavily
from the existing blockpull command.

* tools/virsh-domain.c (cmdBlockCommit): New function.
* tools/virsh.pod (blockcommit): Document it.

Add a new member variable map_len to store map len of bitmap.
and rename size to max_bit accordingly.

rename virBitmapAlloc to virBitmapNew.

Some combinations of filtering flags produce no result. This patch tries
to clarify this.

New command node-memory-tune to get/set the node memory parameters,
only two parameters are allowed to set (pages_to_scan, and sleep_millisecs,
see documents in this patch for more details).

Example of node-memory-tune's output:

Shared memory:
	pages_to_scan   100
	sleep_millisecs 20
	pages_shared    0
	pages_sharing   0
	pages_unshared  0
	pages_volatile  0
	full_scans      0

This introduces four new options for secret-list, to filter the
returned secrets by whether it's ephemeral or not, and/or by
whether it's private or not.

* tools/virsh-secret.c: (New helper vshSecretSorter,
  vshSecretListFree, and vshCollectSecretList; Use the new
  API for secret-list; error out if flags are specified,
  because there is no way to filter the results when using
  old APIs (no APIs to get the properties (ephemeral, private)
  of a secret yet).

* tools/virsh.pod: Document the 4 new options.

tools/virsh-nwfilter.c:
  * vshNWFilterSorter to sort network filters by name

  * vshNWFilterListFree to free the network filter objects list.

  * vshNWFilterListCollect to collect the network filter objects, trying
    to use new API first, fall back to older APIs if it's not supported.

tools/virsh-nodedev.c:
  * vshNodeDeviceSorter to sort node devices by name

  * vshNodeDeviceListFree to free the node device objects list.

  * vshNodeDeviceListCollect to collect the node device objects, trying
    to use new API first, fall back to older APIs if it's not supported.

  * Change option --cap to accept multiple capability types.

tools/virsh.pod
  * Update document for --cap

This improve helper vshStringToArray to accept const string as
argument instead. To not convert the const string when using
vshStringToArray, and thus avoid motifying it.

And redundant error resetting.

Pushed under trivial rule.

I got an off-list report about a bad diagnostic:
Target network card mac 52:54:00:49:07:ccdoes not match source 52:54:00:49:07:b8

True to form, I've added a syntax check rule to prevent it
from recurring, and found several other offenders.

* cfg.mk (sc_require_whitespace_in_translation): New rule.
* src/conf/domain_conf.c (virDomainNetDefCheckABIStability): Add
space.
* src/esx/esx_util.c (esxUtil_ParseUri): Likewise.
* src/qemu/qemu_command.c (qemuCollectPCIAddress): Likewise.
* src/qemu/qemu_driver.c (qemuDomainSetMetadata)
(qemuDomainGetMetadata): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeNetBridge): Likewise.
* src/rpc/virnettlscontext.c
(virNetTLSContextCheckCertDNWhitelist): Likewise.
* src/vmware/vmware_driver.c (vmwareDomainResume): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc, vboxAttachDrives):
Avoid false negatives.
* tools/virsh-domain.c (info_save_image_dumpxml): Reword.
Based on a report by Luwen Su.

tools/virsh-interface.c:
  * vshInterfaceSorter to sort interfaces by name

  * vshInterfaceListFree to free the interface objects list.

  * vshInterfaceListCollect to collect the interface objects, trying
    to use new API first, fall back to older APIs if it's not supported.

* tools/virsh-network.c: s/MATCH/VSH_MATCH/

tools/virsh-network.c:
  * vshNetworkSorter to sort networks by name

  * vshNetworkListFree to free the network objects list.

  * vshNetworkListCollect to collect the network objects, trying
    to use new API first, fall back to older APIs if it's not supported.

  * New options --persistent, --transient, --autostart, --no-autostart,
    for net-list, and new field 'Persistent' for its output.

tools/virsh.pod:
  * Add documents for the new options.

tools/virsh-volume.c:
  * vshStorageVolSorter to sort storage vols by name

  * vshStorageVolumeListFree to free the volume objects list

  * vshStorageVolumeListCollect to collect the volume objects, trying
    to use new API first, fall back to older APIs if it's not supported.

When setting the cpu tunables in virsh you are able to update only a
subset of them. Virsh while doing the update updated all of the
tunables, changed ones with new values and unchanged with old ones.
This is unfortunate as it:
a) might overwrite some other change by a race condition (unprobable)
b) fails with range checking as some of the old values saved might be
   out of range

This patch changes the update procedure so that only the changed value
is updated on the host.

This patch also fixes a very unprobable memory leak if the daemon would
return a string tunable parameter, as the typed parameter array was not
cleared.

Which is lost by commit 93a346d3.

A last minute rename in commit fc122e1a to virsh.h was not properly
reflected when rebasing virsh-pool.c in commit 93a346d3.

* tools/virsh-pool.c (vshStoragePoolListCollect): Use VSH_MATCH,
not MATCH.

tools/virsh-pool.c:
  * vshStoragePoolSorter to sort the pool list by pool name.

  * struct vshStoragePoolList to present the pool list, pool info
    is collected by list->poolinfo if 'details' is specified by
    user.

  * vshStoragePoolListFree to free the pool list

  * vshStoragePoolListCollect to collect the pool list, new API
    virStorageListAllPools is tried first, if it's not supported,
    fall back to older APIs.

  * New options --persistent, --transient, --autostart, --no-autostart
    and --type for pool-list. --persistent or --transient is to filter
    the returned pool list by whether the pool is persistent or not.
    --autostart or --no-autostart is to filter the returned pool list
    by whether the pool is autostarting or not. --type is to filter
    the pools by pool types. E.g.

    % virsh pool-list --all --persistent --type dir,disk

tools/virsh.pod:
   * Add documentations for the new options.

Move definition of MATCH from virsh-domain-monitor.c into
virsh.h, and rename it as VSH_MATCH for further use.

* tools/virsh-domain-monitor.c: Change MATCH into VSH_MATCH
* tools/virsh.h: Define VSH_MATCH

The storage pool's management doesn't relate with a domain, it
probably was an intention, but not achieved yet. And the fact
is only active pools are listed by default.

tools/virsh.c: New helper function vshStringToArray.
tools/virsh.h: Declare vshStringToArray.
tools/virsh-domain.c: use the helper in cmdUndefine.

Yesterday's commit 15d2c9fa pointed out that virsh was still using
localtime(), which is not thread-safe, even though virsh is
definitely multi-threaded.  Even if we only ever triggered it from
one thread, it's better safe than sorry for maintenance purposes.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_nonreentrant):
Tighten the rule.
* tools/virsh.c (vshOutputLogFile): Avoid localtime.
(vshEditWriteToTempFile, vshEditReadBackFile, cmdCd, cmdPwd)
(vshCloseLogFile): Avoid strerror.
* tools/console.c (vshMakeStdinRaw): Likewise.
* tools/virsh-domain.c (vshGenFileName): Fix spacing in previous
patch.

On 09/04/2012 08:20 AM, Eric Blake wrote:
> tv_sec is required by POSIX to be
> of type time_t; so this is a bug in the OpenBSD header
> [for declaring it as long]

Most likely this problem arose because of the patch I pushed
in gnulib commit e07d7c40f3ca5ec410cf5aa6fa03cfe51e712039.
Previously, gnulib required timeval's tv_sec to be
the same size as time_t.  But now, it requires only that
tv_sec be big enough to hold a time_t.

This patch was needed for Emacs.  Without the patch, gnulib
replaced struct timeval on OpenBSD, and this messed up
utimens.c, and Emacs wouldn't build.

Alternatively, gnulib could substitute its own struct timeval
for the system's, wrapping every struct timeval-using function
(gettimeofday, futimesat, futimes, lutimes, etc.  That'd be
more work, though.  And it would introduce some performance
issues with gettimeofday, which is supposed to be fast.

I've been trying to get away from using struct timeval,
and to use the higher-resolution struct timespec instead,
so messing with these obsolescent interfaces has been
lower priority for me.  But if someone wants to take the
more-ambitious approach that'd be fine, I expect.

For this particular case, though, how about if we avoid
the problem entirely?  libvirt doesn't need to use struct
timeval here at all.  It makes libvirt smaller and probably
faster, and it ports to OpenBSD without messing with gnulib.

Now that vshCommandRun() checks for the connection automaticaly, remove
all of the redundant checks in the code.

vshConnectionUsability() no longer needs to be exported and this patch
marks it static.