Host tcp4/tcp6 ports is a global resource thus we need to make
port accounting also global or we have issues described in [1] when
port allocator ranges of different instances are overlapped (which
is by default for qemu for example).

Let's have only one global port allocator object that take care
of the entire ports range (0 - 65535) and introduce port range object
for clients to specify desired auto allocation band.

[1] https://www.redhat.com/archives/libvir-list/2017-December/msg00600.htmlSigned-off-by: NNikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>

Cast away enum type for libxl scheduler constants since we don't want to
cover all of them and don't want build to break when new ones are added.
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Currently virt drivers will call directly into the network driver impl
to allocate domain interface devices where type=network. This introduces
a callback system to allow us to decouple the virt drivers from the
network driver.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

A Xen domain0 is better described as a persistent domain. Mark it
as such during intialization.

Use bool instead of an int.

Right-aligning backslashes when defining macros or using complex
commands in Makefiles looks cute, but as soon as any changes is
required to the code you end up with either distractingly broken
alignment or unnecessarily big diffs where most of the changes
are just pushing all backslashes a few characters to one side.

Generated using

  $ git grep -El '[[:blank:]][[:blank:]]\\$' | \
    grep -E '*\.([chx]|am|mk)$$' | \
    while read f; do \
      sed -Ei 's/[[:blank:]]*[[:blank:]]\\$/ \\/g' "$f"; \
    done
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

All APIs which expect a list of CPU models supported by hypervisors were
switched from char **models and int models to just accept a pointer to
virDomainCapsCPUModels object stored in domain capabilities. This avoids
the need to transform virDomainCapsCPUModelsPtr into a NULL-terminated
list of model names and also allows the various cpu driver APIs to
access additional details (such as its usability) about each CPU model.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1497396

In 0d3d020b I've added capability to accept MAC addresses
for the API too. However, the implementation was faulty. It needs
to lookup the corresponding interface in the domain definition
and pass the ifname instead of MAC address.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1497396

The other APIs accept both, ifname and MAC address. There's no
reason virDomainInterfaceStats can't do the same.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

Every caller reports the error themselves. Might as well move it
into the function and thus unify it.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1497410

The comment in virNetDevTapInterfaceStats() implementation for
Linux states that packets transmitted by domain are received by
the host and vice versa. Well, this is true but not for all types
of interfaces. For instance, for macvtaps when TAP device is
hooked right onto a physical device any packet that domain sends
looks also like a packet sent to the host. Therefore, we should
allow caller to chose if the stats returned should be straight
copy or swapped.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

In the past we updated host-model CPUs with host CPU data by adding a
model and features, but keeping the host-model mode. And since the CPU
model is not normally formatted for host-model CPU defs, we had to pass
the updateCPU flag to the formatting code to be able to properly output
updated host-model CPUs. Libvirt doesn't do this anymore, host-model
CPUs are turned into custom mode CPUs once updated with host CPU data
and thus there's no reason for keeping the hacks inside CPU XML
formatters.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The implementation of virConnectBaselineCPU may be different for each
hypervisor. Thus it shouldn't really be implmented in the cpu code.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Using a variable named 'stat' clashes with the system function
'stat()' causing compiler warnings on some platforms:

libxl/libxl_driver.c: In function 'libxlDomainBlockStatsVBD':
libxl/libxl_driver.c:5387: error: declaration of 'stat' shadows a global declaration [-Wshadow]
/usr/include/sys/stat.h:455: error: shadowed declaration is here [-Wshadow]
Signed-off-by: NGeorge Dunlap <george.dunlap@citrix.com>
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>

Rewrite virConfReadMem to take a null-terminated string.
All the callers were calling strlen on it anyway.

If a remote call fails during event registration (more than likely from
a network failure or remote libvirtd restart timed just right), then when
calling the virObjectEventStateDeregisterID we don't want to call the
registered @freecb function because that breaks our contract that we
would only call it after succesfully returning.  If the @freecb routine
were called, it could result in a double free from properly coded
applications that free their opaque data on failure to register, as seen
in the following details:

    Program terminated with signal 6, Aborted.
    #0  0x00007fc45cba15d7 in raise
    #1  0x00007fc45cba2cc8 in abort
    #2  0x00007fc45cbe12f7 in __libc_message
    #3  0x00007fc45cbe86d3 in _int_free
    #4  0x00007fc45d8d292c in PyDict_Fini
    #5  0x00007fc45d94f46a in Py_Finalize
    #6  0x00007fc45d960735 in Py_Main
    #7  0x00007fc45cb8daf5 in __libc_start_main
    #8  0x0000000000400721 in _start

The double dereference of 'pyobj_cbData' is triggered in the following way:

    (1) libvirt_virConnectDomainEventRegisterAny is invoked.
    (2) the event is successfully added to the event callback list
        (virDomainEventStateRegisterClient in
        remoteConnectDomainEventRegisterAny returns 1 which means ok).
    (3) when function remoteConnectDomainEventRegisterAny is hit,
        network connection disconnected coincidently (or libvirtd is
        restarted) in the context of function 'call' then the connection
        is lost and the function 'call' failed, the branch
        virObjectEventStateDeregisterID is therefore taken.
    (4) 'pyobj_conn' is dereferenced the 1st time in
        libvirt_virConnectDomainEventFreeFunc.
    (5) 'pyobj_cbData' (refered to pyobj_conn) is dereferenced the
         2nd time in libvirt_virConnectDomainEventRegisterAny.
    (6) the double free error is triggered.

Resolve this by adding a @doFreeCb boolean in order to avoid calling the
freeCb in virObjectEventStateDeregisterID for any remote call failure in
a remoteConnect*EventRegister* API. For remoteConnect*EventDeregister* calls,
the passed value would be true indicating they should run the freecb if it
exists; whereas, it's false for the remote call failure path.

Patch based on the investigation and initial patch posted by
fangying <fangying1@huawei.com>.

So far our code is full of the following pattern:

  dom = virGetDomain(conn, name, uuid)
  if (dom)
      dom->id = 42;

There is no reasong why it couldn't be just:

  dom = virGetDomain(conn, name, uuid, id);

After all, client domain representation consists of tuple (name,
uuid, id).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Tunnelled migration doesn't require any extra network connections
beside the libvirt daemon.  It's capable of strong encryption and the
default option of openstack-nova.

This patch adds the tunnelled migration(Tunnel3params) support to
libxl.  On the source side, the data flow is:

 * libxlDoMigrateSend() -> pipe libxlTunnel3MigrationFunc() polls pipe
 * out and then write to dest stream.

While on the destination side:
 * Stream -> pipe -> 'recvfd of libxlDomainStartRestore'

The usage is the same as p2p migration, execpt adding one extra
'--tunnelled' to the libvirt p2p migration command.
Signed-off-by: NBob Liu <bob.liu@oracle.com>
Signed-off-by: NJoao Martins <joao.m.martins@oracle.com>

When the libxl driver is initialized, it creates a virDomainDef
object for dom0 and adds it to the list of domains. Total memory
for dom0 was being set from the max_memkb field of libxl_dominfo
struct retrieved from libxl, but this field can be set to
LIBXL_MEMKB_DEFAULT (~0ULL) if dom0 maximum memory has not been
explicitly set by the user.

This patch adds some simple parsing of the Xen commandline,
looking for a dom0_mem parameter that also specifies a 'max' value.
If not specified, dom0 maximum memory is effectively all physical
host memory.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

The libxl driver reports different values of maximum memory depending
on state of a domain. If inactive, maximum memory value is reported
correctly. When active, maximum memory is derived from max_pages value
returned by the XEN_SYSCTL_getdomaininfolist sysctl operation. But
max_pages can be changed by toolstacks and does not necessarily
represent the maximum memory a domain can use during its active
lifetime.

A better location for determining a domain's maximum memory is the
/local/domain/<id>/memory/static-max node in xenstore. This value
is set from the libxl_domain_build_info.max_memkb field when creating
the domain. Currently it cannot be changed nor can its value be
exceeded by a balloon operation. From libvirt's perspective, always
reporting maximum memory with virDomainDefGetMemoryTotal() will produce
the same results as reading the static-max node in xenstore.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

When a user does not explicitly set a <driver> in the disk config,
libvirt defers selection of a default to libxl. This approach works
fine when starting a domain with such configuration or attaching a
disk to a running domain. But when detaching such a disk, libxl
will fail with "unrecognized disk backend type: 0". libxl makes no
attempt to recalculate a default backend (driver) on detach and
simply fails when uninitialized.

This patch updates the libvirt disk config with the backend selected
by libxl when starting a domain or attaching a disk to a running
domain. Another benefit of this approach is that the live XML is
also updated with the backend driver selected by libxl.

When starting a domian, a libxl_domain_config object is created from
virDomainDef. Any virDomainDiskDef devices with a format of
VIR_STORAGE_FILE_NONE are mapped to LIBXL_DISK_FORMAT_RAW in the
corresponding libxl_disk_device, but the virDomainDiskDef format is
never updated to reflect the change.

A better place to set a default format for disk devices is the
device post-parse callback, ensuring the virDomainDiskDef object
reflects the default format.

The typical pattern when calling libxl functions that populate a
structure is

  libxl_foo foo;
  libxl_foo_init(&foo);
  libxl_get_foo(ctx, &foo);
  ...
  libxl_foo_dispose(&foo);

Fix several instances of libxl_physinfo missing the init and
dispose calls.
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

libxl doesn't provide a way to write one log for each domain. Thus
we need to demux the messages. If our logger doesn't know to which
domain to attribute a message, then it will write it to the default
log file.

Starting with Xen 4.9 (commit f9858025 and following), libxl will
write the domain ID in an easy to grab manner. The logger introduced
by this commit will use it to demux the libxl log messages.

Thanks to the default log file, this logger will also work with older
versions of Xen.

The libxl driver already supports getting maximum vcpu count via
libxlDomainGetVcpusFlags, allowing to trivially implement
virDomainGetMaxVcpus.

libvirt libxl picks its own default with respect to the default NIC
to use. libxlMakeNic is the one responsible for this and on boot it
picks LIBXL_NIC_TYPE_VIF_IOEMU for HVM domains such that it accomodates
both PV and emulated one. The good behaving guest at boot will then
select the pv and unplug the emulated device.

Now, on HVM when attaching an interface it will pick the same default
that is LIBXL_NIC_TYPE_VIF_IOEMU which as a result will fail the attach
(see xen commit 32e9d0f ("libxl: nic type defaults to vif in hotplug for
hvm guest"). Xen doesn't yet support the hotplug of emulated devices,
but we don't want to rule out that case either, which might get support
in the future. Hence we simply reverse the defaults when we are
attaching the interface which allows libvirt to prefer the PV nic first
without adding "model='netfront'" following the same pattern as above
commit. Also to avoid ruling out the emulated one we set to
LIBXL_NIC_TYPE_IOEMU when setting a model type that is not 'netfront'.
Signed-off-by: NJoao Martins <joao.m.martins@oracle.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

This is just a code move of virstat.c to virnetdevtap.c

In preparation to the code move to virnetdevtap.c, this change:

* renames virNetInterfaceStats to virNetDevTapInterfaceStats
* changes 'path' to 'ifname', to use the same vocable as other
  method in virnetdevtap.c.
* Add the attributes checker

New line character in name of domain is now forbidden because it
mess virsh output and can be confusing for users.
Validation of name is done in drivers, after parsing XML to avoid
problems with dissappeared domains which was already created with
new-line char in name.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Change the virDomainChrDef to use a pointer to 'source' and allocate
that pointer during virDomainChrDefNew.

This has tremendous "fallout" in the rest of the code which mainly
has to change source.$field to source->$field.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

This function for some weird reason returns integer instead of
virDomainNetType type. It is important to return the correct type
so that we know what values we can expect.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This way we get reference counting and we can get rid of locking
function.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Due to a copy and paste error, the scheduler 'cap' parameter
was over-writing the 'weight' parameter when preparing the
return parameters in libxlDomainGetSchedulerParametersFlags.
As a result, the scheduler weight was never shown when getting
schedinfo and setting the weight failed as well

virsh  schedinfo testvm
Scheduler      : credit
cap            : 0

virsh  schedinfo testvm --cap 50 --weight 500
Scheduler      : credit
error: invalid scheduler option: weight

The obvious fix is to assign the 'caps' parameter to the correct
item in the parameter list.
Reported-by: NVolo M. <vm@vovs.net>

And allow libxl to handle channel element which creates a Xen
console visible to the guest as a low-bandwitdh communication
channel. If type is PTY we also fetch the tty after boot using
libxl_channel_getinfo to fetch the tty path. On socket case,
we autogenerate a path if not specified in the XML. Path autogenerated
is slightly different from qemu driver: qemu stores also on
"channels/target" but it creates then a directory per domain with
each channel target name. libxl doesn't appear to have a clear
definition of private files associated with each domain, so for
simplicity we do it slightly different. On qemu each autogenerated
channel goes like:

channels/target/<domain-name>/<target name>

Whereas for libxl:

channels/target/<domain-name>-<target name>

Should note that if path is not specified it won't persist,
existing only on live XML, unless user had initially specified it.
Since support for libxl channels only came on Xen >= 4.5 we therefore
need to conditionally compile it with LIBXL_HAVE_DEVICE_CHANNEL.

After this patch and having a qemu guest agent:
 $ cat domain.xml | grep -a1 channel | head -n 5 | tail -n 4
 <channel type='unix'>
   <source mode='bind' path='/tmp/channel'/>
   <target type='xen' name='org.qemu.guest_agent.0'/>
 </channel>

 $ virsh create domain.xml
 $ echo '{"execute":"guest-network-get-interfaces"}' | socat
 stdio,ignoreeof  unix-connect:/tmp/channel

 {"execute":"guest-network-get-interfaces"}
 {"return": [{"name": "lo", "ip-addresses": [{"ip-address-type": "ipv4",
 "ip-address": "127.0.0.1", "prefix": 8}, {"ip-address-type": "ipv6",
 "ip-address": "::1", "prefix": 128}], "hardware-address":
 "00:00:00:00:00:00"}, {"name": "eth0", "ip-addresses":
 [{"ip-address-type": "ipv4", "ip-address": "10.100.0.6", "prefix": 24},
 {"ip-address-type": "ipv6", "ip-address": "fe80::216:3eff:fe40:88eb",
 "prefix": 64}], "hardware-address": "00:16:3e:40:88:eb"}, {"name":
 "sit0"}]}
Signed-off-by: NJoao Martins <joao.m.martins@oracle.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

We want to pass the proper opaque pointer instead of NULL to
virDomainDefParse and subsequently virDomainDefParseNode too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Both cpuCompare* APIs are renamed to virCPUCompare*. And they should now
work for any guest CPU definition, i.e., even for host-passthrough
(trivial) and host-model CPUs. The implementation in x86 driver is
enhanced to provide a hint about -noTSX Broadwell and Haswell models
when appropriate.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Akin to previous commit but for "virsh cpu-baseline" which
computes a baseline CPU for a set of host cpu elements.
Signed-off-by: NJoao Martins <joao.m.martins@oracle.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Implement support for "virsh cpu-compare" so that we can calculate
common cpu element between a pool of hosts, which had a requirement
of providing host cpu description.
Signed-off-by: NJoao Martins <joao.m.martins@oracle.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add support for multi serial devices, after this patch virsh can be used to
connect different serial devices of running domains. E.g.
vish # console <xxx> --devname serial<xxx>

Note:
This depends on a xen/libxl bug fix to have libxl_console_get_tty(...) correctly
returning the tty path (as opposed to always returning the first one).
[0] https://lists.xen.org/archives/html/xen-devel/2016-08/msg00438.htmlSigned-off-by: NBob Liu <bob.liu@oracle.com>