As pointed out by jtomko in his review of the IOThreads pinning code:

http://www.redhat.com/archives/libvir-list/2015-March/msg00495.html

there are some comments sprinkled in indicating IOThreads were using
the same structure as the VcpuPin code...

This is the first patch of a few that will change the virDomainVcpuPin*
structures and code to just virDomainPin* - starting with the data
structure naming...

As there are two possible approaches to define a domain's memory size -
one used with legacy, non-NUMA VMs configured in the <memory> element
and per-node based approach on NUMA machines - the user needs to make
sure that both are specified correctly in the NUMA case.

To avoid this burden on the user I'd like to replace the NUMA case with
automatic totaling of the memory size. To achieve this I need to replace
direct access to the virDomainMemtune's 'max_balloon' field with
two separate getters depending on the desired size.

The two sizes are needed as:
1) Startup memory size doesn't include memory modules in some
hypervisors.
2) After startup these count as the usable memory size.

Note that the comments for the functions are future aware and document
state that will be present after a few later patches.

It will not be possible to detach such device later. Also improve
logging in such cases.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

A helper that never returns an error and treats bits out of bitmap range
as false.

Use it everywhere we use ignore_value on virBitmapGetBit, or loop over
the bitmap size.

In the old days of a global driver lock, it was necessary to unlock
the driver after a domain restore operation.  When the global lock
was removed from the driver, some remnants were left behind in
libxlDomainRestoreFlags.  Remove this unneeded (and incorrect) code.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Otherwise domainCreateWithFlags via remote returns -1 as domid.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

For stateless, client side drivers, it is never correct to
probe for secondary drivers. It is only ever appropriate to
use the secondary driver that is associated with the
hypervisor in question. As a result the ESX & HyperV drivers
have both been forced to do hacks where they register no-op
drivers for the ones they don't implement.

For stateful, server side drivers, we always just want to
use the same built-in shared driver. The exception is
virtualbox which is really a stateless driver and so wants
to use its own server side secondary drivers. To deal with
this virtualbox has to be built as 3 separate loadable
modules to allow registration to work in the right order.

This can all be simplified by introducing a new struct
recording the precise set of secondary drivers each
hypervisor driver wants

struct _virConnectDriver {
    virHypervisorDriverPtr hypervisorDriver;
    virInterfaceDriverPtr interfaceDriver;
    virNetworkDriverPtr networkDriver;
    virNodeDeviceDriverPtr nodeDeviceDriver;
    virNWFilterDriverPtr nwfilterDriver;
    virSecretDriverPtr secretDriver;
    virStorageDriverPtr storageDriver;
};

Instead of registering the hypervisor driver, we now
just register a virConnectDriver instead. This allows
us to remove all probing of secondary drivers. Once we
have chosen the primary driver, we immediately know the
correct secondary drivers to use.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The path to the pty of a Xen PV console is set only in
virDomainOpenConsole. But this is done too late. A call to
virDomainGetXMLDesc done before OpenConsole will not have the path to
the pty, but a call after OpenConsole will.

e.g. of the current issue.
Starting a domain with '<console type="pty"/>'
Then:
virDomainGetXMLDesc():
  <devices>
    <console type='pty'>
      <target type='xen' port='0'/>
    </console>
  </devices>
virDomainOpenConsole()
virDomainGetXMLDesc():
  <devices>
    <console type='pty' tty='/dev/pts/30'>
      <source path='/dev/pts/30'/>
      <target type='xen' port='0'/>
    </console>
  </devices>

The patch intend to have the TTY path on the first call of GetXMLDesc.
This is done by setting up the path at domain start up instead of in
OpenConsole.

https://bugzilla.redhat.com/show_bug.cgi?id=1170743Signed-off-by: NAnthony PERARD <anthony.perard@citrix.com>

The virDomainDefineXMLFlags and virDomainCreateXML APIs both
gain new flags allowing them to be told to validate XML.
This updates all the drivers to turn on validation in the
XML parser when the flags are set

Now that xenconfig supports parsing and formatting Xen's
XL config format, integrate it into the libxl driver's
connectDomainXML{From,To}Native functions.
Signed-off-by: NKiarie Kahurani <davidkiarie4@gmail.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

The virDomainDefParse* and virDomainDefFormat* methods both
accept the VIR_DOMAIN_XML_* flags defined in the public API,
along with a set of other VIR_DOMAIN_XML_INTERNAL_* flags
defined in domain_conf.c.

This is seriously confusing & error prone for a number of
reasons:

 - VIR_DOMAIN_XML_SECURE, VIR_DOMAIN_XML_MIGRATABLE and
   VIR_DOMAIN_XML_UPDATE_CPU are only relevant for the
   formatting operation
 - Some of the VIR_DOMAIN_XML_INTERNAL_* flags only apply
   to parse or to format, but not both.

This patch cleanly separates out the flags. There are two
distint VIR_DOMAIN_DEF_PARSE_* and VIR_DOMAIN_DEF_FORMAT_*
flags that are used by the corresponding methods. The
VIR_DOMAIN_XML_* flags received via public API calls must
be converted to the VIR_DOMAIN_DEF_FORMAT_* flags where
needed.

The various calls to virDomainDefParse which hardcoded the
use of the VIR_DOMAIN_XML_INACTIVE flag change to use the
VIR_DOMAIN_DEF_PARSE_INACTIVE flag.

Make sure every virt driver implements virDomainDefineXMLFlags
by adding a trivial passthrough from the existing impl with
no flags set.

This reverts commit 4f524212.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Now that xenconfig supports parsing and formatting Xen's
XL config format, integrate it into the libxl driver's
connectDomainXML{From,To}Native functions.
Signed-off-by: NKiarie Kahurani <davidkiarie4@gmail.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

This patch contains three domain cleanup improvements in the migration
finish phase, ensuring a domain is properly disposed when a failure is
detected or the migration is cancelled.

The check for virDomainObjIsActive is moved to libxlDomainMigrationFinish,
where cleanup can occur if migration failed and the domain is inactive.

The 'cleanup' label was missplaced in libxlDomainMigrationFinish, causing
a migrated domain to remain in the event of an error or cancelled migration.

In cleanup, the domain was not removed from the driver's list of domains.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Moving data reception of the perform phase of migration to a
thread introduces a race with the finish phase, where checking
if the domain is active races with the thread finishing the
perform phase.  The race is easily solved by acquiring a job in
the finish phase, which must wait for the perform phase job to
complete.

While wrapping the finish phase in a job, noticed the virDomainObj
was being unlocked in a callee - libxlDomainMigrationFinish.  Move
the unlocking to libxlDomainMigrateFinish3Params, where the lock
is acquired.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

C guarantees that static variables are zero-initialized.  Some older
compilers (and also gcc -fno-zero-initialized-in-bss) create larger
binaries if you explicitly zero-initialize a static variable.

* src/libxl/libxl_driver.c: Fix initialization.
* src/lxc/lxc_controller.c: Likewise.
* src/openvz/openvz_util.c (openvzKBPerPages): Likewise.
* src/phyp/phyp_driver.c: Likewise.
* src/remote/remote_driver.c: Likewise.
* src/test/test_driver.c: Likewise.
* src/uml/uml_driver.c: Likewise.
* src/vbox/vbox_XPCOMCGlue.c: Likewise.
* src/vbox/vbox_tmpl.c: Likewise.
* src/xen/xen_driver.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

To prepare for introducing a single global driver, rename the
virDriver struct to virHypervisorDriver and the registration
API to virRegisterHypervisorDriver()

There is no need to acquire the driver-wide lock in
libxlDomainDefineXML.  When switching to jobs in the libxl
driver, most driver-wide locks were removed.  The locking here
was preserved since I mistakenly thought virDomainObjListAdd
needed protection.  This is not the case, so remove the
unnecessary locking.

Add a check of the return for virDomainHostdevInsert() like every
other call.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Test suites using the port allocator don't want to have different
behaviour depending on whether a port is in use on the host. Add
a VIR_PORT_ALLOCATOR_SKIP_BIND_CHECK which test suites can use
to skip the bind() test. The port allocator will thus only track
ports in use by the test suite process itself. This is fine when
using the port allocator to generate guest configs which won't
actually be launched
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Create a separate typedef for the hostdev union data describing PCI.
Then adjust the code to use the new pointer

Code logic in libxlDomainAttachDeviceFlags and libxlDomainDetachDeviceFlags
is wrong with return value in error cases.

'ret' was being set to 0 if 'flags & VIR_DOMAIN_DEVICE_MODIFY_CONFIG' was
false. Then if something like virDomainDeviceDefParse() failed in the
VIR_DOMAIN_DEVICE_MODIFY_LIVE logic, the error would be reported but the
function would return success.
Signed-off-by: NChunyan Liu <cyliu@suse.com>

Add code to support attach/detaching a network device.
Signed-off-by: NChunyan Liu <cyliu@suse.com>

Missing HOSTDEV type in libxlDomainDetachDeviceConfig. Add it.
Signed-off-by: NChunyan Liu <cyliu@suse.com>

During migration, the libxl driver starts a modify job in the
begin phase, ending the job in the confirm phase.  This is
essentially VIR_MIGRATE_CHANGE_PROTECTION semantics, but the
driver does not support that flag.  Without CHANGE_PROTECTION
support, the job would never be terminated in error conditions
where migrate confirm phase is not executed.  Further attempts
to modify the domain would result in failure to acquire a job
after LIBXL_JOB_WAIT_TIME.

Similar to the qemu driver, end the job in the begin phase.
Protecting the domain object across all phases of migration can
be done in a future patch adding CHANGE_PROTECTION support.

In libxlDomainMigrationPrepare(), a new virDomainObj is created
from the incoming domain def and added to the driver's domain
list, but never removed if there are subsequent failures during
the prepare phase.

targethost# virsh list --all

sourcehost# virsh migrate --live dom xen+ssh://targethost/system
   error: operation failed: Fail to create socket for incoming migration.

targethost# virsh list --all
error: Failed to list domains
error: name in virGetDomain must not be NULL

After adding code to remove the domain on prepare failure, noticed
that libvirtd crashed due to double free of the virDomainDef.  Similar
to the qemu driver, pass a pointer to virDomainDefPtr so it can be set
to NULL once a virDomainObj is created from it.

Replace:
if (virBufferError(&buf)) {
    virBufferFreeAndReset(&buf);
    virReportOOMError();
    ...
}

with:
if (virBufferCheckError(&buf) < 0)
    ...

This should not be a functional change (unless some callers
misused the virBuffer APIs - a different error would be reported
then)

So far, we only report an error if formatting the siblings bitmap
in NUMA topology fails.

Be consistent and always report error in virCapabilitiesFormatXML.

libxl does not support save, restore, or migrate on all architectures,
notably ARM.  Detect whether libxl supports these operations using
LIBXL_HAVE_NO_SUSPEND_RESUME.  If not supported, drop advertisement of
<migration_features>.

Found by Ian Campbell while improving Xen's OSSTEST infrastructure

http://lists.xen.org/archives/html/xen-devel/2014-06/msg02171.html

Commit 9b8d6e1e missed updating the libvirt version which
introduced support for domainMigrate*3Params functions.

This simplifies the usage in {libxl,qemu}DomainGetNumaParameters
and it's needed for consistent error reporting in virBitmapFormat.

Also remove the forgotten ATTRIBUTE_NONNULL marker.

This patch adds initial migration support to the libxl driver,
using the VIR_DRV_FEATURE_MIGRATION_PARAMS family of migration
functions.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

To allow the test suite to creat the XML option object,
move the virDomainXMLOptionNew call into a libxlCreateXMLConf
method.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add support for the VIR_DOMAIN_REBOOT_PARAVIRT flag in
libxlDomainReboot().

Add support for VIR_DOMAIN_SHUTDOWN_PARAVIRT and
VIR_DOMAIN_SHUTDOWN_ACPI_POWER_BTN flags in
libxlDomainShutdownFlags().

With xend on the way out, installations may not even have
/usr/sbin/xend, which results in the following error when the
drivers are probed

2014-04-28 18:21:19.271+0000: 22129: error : virCommandWait:2426 :
internal error: Child process (/usr/sbin/xend status) unexpected exit
status 127: libvirt:  error : cannot execute binary /usr/sbin/xend:
No such file or directory

Check for existence of /usr/sbin/xend before trying to run it with
the 'status' option.

Currently the driver only exposes the ability to connect to the serial console
of a Xen guest, which doesn't work for a PV guest. Since for an HVM guest the
serial devices are duplicated as consoles it is sufficient to just use the
console devices unconditionally.

Tested with the following bit of config XML:

<domain type='xen'>
  ...
  <devices>
    <console type='pty'>
      <target type='xen'/>
    </console>
  </devices>
</domain>

I have observed and tested this on ARM but I believe it also applies to x86 PV
guests.
Signed-off-by: NIan Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Clark Laughlin <clark.laughlin@linaro.org>