There's lot more to document about the nodedev driver, besides PCI and
SR-IOV (even this might need to be extended), but let's start small-ish
and at least have a page for it linked from the drivers.html.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1439132

During 'matrix' testing of all possible combinations I found that if
device is formated with "gpt" first, then an attempt is made to format
using "mac", a startup will fail.

Deeper analysis by Peter Krempa indicates that the "mac" table fits
into the first block on the disk. Since the GPT disklabel is stored
at LBA address 1 it is not overwritten at all. Thus it's apparent that
the (blkid) detection tool then prefers GPT over a older disklabel.

The GPT disklabel has also a secondary copy at the last LBA of the disk.

So, follow the same logic as the logical pool in clearing a 1MB swath
at the beginning and end of the device to avoid potential issues with
larger sector sizes for the device.

Also fixed a minor formatting nit in virStorageBackendDeviceIsEmpty call.

Since a sector size may be larger than 512 bytes, let's just increase
the size to wipe to 1MB rather than 2KB

Rather than open code it, use the new function which uses the wipe algorithm
in order to zero the front and tail of the partition.

Create a wrapper/helper that can be used to call the storage backend
wipe helper - storageBackendVolWipeLocalFile for future use by logical
and disk backends to clear out the partition table rather than having
each open code the same algorithm.

Add bool 'zero_end' and logic that would allow a caller to wipe specific
portions of a target device either from the beginning (the default) or
from the end when zero_end is true.

This will allow for this code to wipe out partition table information
from a device.

There were a few bugs in keycodemap tool that broke it
when run on python circa 2.7.5 or older, which affected
RHEL builds.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A long time ago we imported the keymaps.csv file from GTK-VNC so we
can do conversions between keycode sets. Meanwhile lots of bug fixes
have gone into this CSV file and libvirt hasn't kept in sync. The
keymaps.csv file and associated generator script has been pulled out
of GTK-VNC into a dedicated GIT repo for use as a submodule. This
allows GTK-VNC, SPICE-GTK, QEMU and libvirt to share the same master
database and tools and pushing updates merely requires a submodule
commit update as with gnulib.

The test suite is updated to cover some extra boundary conditions.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Shorten the time needed to keep the list lock and alter the cleanup
path to be more of an error path.

Utilize the the virObjectListFree function to handle the calls for
virObjectUnref on each list element and the VIR_FREE of the list
instead of open coding it.

Change the name of the virHashForEach callback to match the name
of the Export function with the Callback added onto it.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Rather than have caller need to do it, have the object returned locked.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Alter the prototypes to use the newer formatting style
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Commit id 'bb1fba62' neglected to set when creating the function.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Commit id '865f479d' altered the logic to use a common test*ObjFindByName
helpers which would lock/unlock the test driver; however, a few cleanup paths
in that cleanup missed removing the Unlock, so remove it now.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Based upon an idea and some research by Wang King <king.wang@huawei.com>
and xinhua.Cao <caoxinhua@huawei.com>.

Since we're assigning the 'client' to our callback event lookaside list,
it's imperative that we grab a reference to the object; otherwise, when
the object is unref'd during virNetServerProcessClients when it's determined
that the virNetServerClientIsClosed and the memory is free'd before perhaps
the object event state callbacks are run.  When a virObjectLock() is run,
before sending the message the following trace occurs;

    #0  0x00007fda223d66d8 in virClassIsDerivedFrom
        (klass=0xdeadbeef, parent=0x7fda24c81b40)
         at util/virobject.c:169
    #1  0x00007fda223d6a1e in virObjectIsClass
        (anyobj=anyobj@entry=0x7fd9e575b400, klass=<optimized out>)
         at util/virobject.c:365
    #2  0x00007fda223d6a44 in virObjectLock
        (anyobj=0x7fd9e575b400)
        at util/virobject.c:317
    #3  0x00007fda22507f71 in virNetServerClientSendMessage
        (client=client@entry=0x7fd9e575b400, msg=msg@entry=0x7fd9ec30de90)
        at rpc/virnetserverclient.c:1422
    #4  0x00007fda230d714d in remoteDispatchObjectEventSend
        (client=0x7fd9e575b400, program=0x7fda24c844e0, procnr=348,
         proc=0x7fda2310e5e0 <xdr_remote_domain_event_callback_tunable_msg>,
         data=0x7ffc3857fdb0)
        at remote.c:3803
    #5  0x00007fda230dd71b in remoteRelayDomainEventTunable
        (conn=<optimized out>, dom=0x7fda27cd7660, params=0x7fda27f3aae0,
         nparams=1,opaque=0x7fd9e6c99e00)
        at remote.c:1033
    #6  0x00007fda224484cb in virDomainEventDispatchDefaultFunc
        (conn=0x7fda27cd0120, event=0x7fda2736ea00, cb=0x7fda230dd610
         <remoteRelayDomainEventTunable>, cbopaque=0x7fd9e6c99e00)
        at conf/domain_event.c:1910
    #7  0x00007fda22446871 in virObjectEventStateDispatchCallbacks
        (callbacks=<optimized out>, callbacks=<optimized out>,
         event=0x7fda2736ea00,state=0x7fda24ca3960)
        at conf/object_event.c:722
    #8  virObjectEventStateQueueDispatch
        (callbacks=0x7fda24c65800, queue=0x7ffc3857fe90, state=0x7fda24ca3960)
        at conf/object_event.c:736
    #9  virObjectEventStateFlush (state=0x7fda24ca3960)
        at conf/object_event.c:814
    #10 virObjectEventTimer (timer=<optimized out>, opaque=0x7fda24ca3960)
        at conf/object_event.c:560
    #11 0x00007fda223ae8b9 in virEventPollDispatchTimeouts ()
        at util/vireventpoll.c:458
    #12 virEventPollRunOnce ()
        at util/vireventpoll.c:654
    #13 0x00007fda223ad1d2 in virEventRunDefaultImpl ()
        at util/virevent.c:314
    #14 0x00007fda225046cd in virNetDaemonRun (dmn=0x7fda24c775c0)
        at rpc/virnetdaemon.c:818
    #15 0x00007fda230d6351 in main (argc=<optimized out>, argv=<optimized out>)
        at libvirtd.c:1623
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Rather than 'n' repetitive code segments, let's create a single macro
which will make the code easier to read.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Freeing the dst is unnecessary if the VIR_STRDUP fails, and therefore
we need to remove the error label as well.

Domain name is duplicated in make_nonnull_domain, but not freed when
virTypedParamsSerialize returns error.

it should be a comparison of modes between new and old devices. So
the argument of the second virDomainNetGetActualDirectMode should be
newdev.
Signed-off-by: NZhiPeng Lu <lu.zhipeng@zte.com.cn>

The goal is twofold: firstly, we want to extend the script so
that it can deal with more than a single git submodule, and
secondly we'd like to reduce the amount of duplicated code.
Moreover, since we're making heavy changes to the code anyway,
we might as well make sure it follows a somewhat consistent
coding style too.

To reduce code duplication, we introduce a new --dry-run
option, which can be used by third parties to figure out
whether calling autogen.sh is necessary or not: this allows
us to get rid of the reimplementation of part of the logic
in cfg.mk and guarantee they'll never get out of sync.

Other changes include: making dirty submodules checking and
cleaning entirely independent of other operations; removing
the use of 'set -e' and handling errors explicitly instead;
better parsing of command line arguments.

Currently, virNetDevSetCoalesce() stub is always returning error. As
it's used by virNetDevTapCreateInBridgePort(), it essentially breaks
bridged networking if coalesce is not supported.

To make it work, relax the stub to trigger error only when its
coalesce argument is not NULL, otherwise report success.

Since we do have this template at hand, why not using it wherever
possible (list of supported pool types and remote access section).
Also, perform some stylistic micro adjustments.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Since we have that information provided by @def which is not a private
object, there is really no need for the variable.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Make the code look cleaner by moving the capability specific bits into
separate functions.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

So udevGetDeviceDetails was one those functions using an enum in a
switch, but since it had a 'default' case, compiler didn't warn about an
unhandled enum. Moreover, the error about an unsupported device type
reported in the default case is unnecessary, since by the time we get
there, udevGetDeviceType (which was called before) already made sure
that any unrecognized device types had been handled properly.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

ka maybe have been freeed in virObjectUnref, application using
virKeepAliveTimer will segfault when unlock ka. We should keep
ka's refs positive before using it.

#0  0x00007fd8f79970e8 in virClassIsDerivedFrom (klass=0xdeadbeef, parent=0x7fd8e8001b80) at util/virobject.c:169
#1  0x00007fd8f799742e in virObjectIsClass (anyobj=anyobj entry=0x7fd8e800b9c0, klass=＜optimized out＞) at util/virobject.c:365
#2  0x00007fd8f79974e4 in virObjectUnlock (anyobj=0x7fd8e800b9c0) at util/virobject.c:338
#3  0x00007fd8f7ac477e in virKeepAliveTimer (timer=＜optimized out＞, opaque=0x7fd8e800b9c0) at rpc/virkeepalive.c:177
#4  0x00007fd8f7e5c9cf in libvirt_virEventInvokeTimeoutCallback () from /usr/lib64/python2.7/site-packages/libvirtmod.so
#5  0x00007fd8ff64db94 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#6  0x00007fd8ff64f1ad in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#7  0x00007fd8ff64d85f in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#8  0x00007fd8ff64d950 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#9  0x00007fd8ff64d950 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#10 0x00007fd8ff64f1ad in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#11 0x00007fd8ff5dc098 in function_call () from /lib64/libpython2.7.so.1.0
#12 0x00007fd8ff5b7073 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#13 0x00007fd8ff5c6085 in instancemethod_call () from /lib64/libpython2.7.so.1.0
#14 0x00007fd8ff5b7073 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#15 0x00007fd8ff648ff7 in PyEval_CallObjectWithKeywords () from /lib64/libpython2.7.so.1.0
#16 0x00007fd8ff67d7e2 in t_bootstrap () from /lib64/libpython2.7.so.1.0
#17 0x00007fd8ff358df3 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fd8fe97d3ed in clone () from /lib64/libc.so.6
Signed-off-by: NYi Wang <wang.yi59@zte.com.cn>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

It was left there after removing a macro it was part of in first
version or so.  Now it will always be NULL.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Commit f4ef3a71 made a variation of virNetDevSetMAC that would return
without logging an error message if errno was set to
EADDRNOTAVAIL. This errno is set by some SRIOV VF drivers (in
particular igbvf) when they fail to set the device's MAC address due
to the PF driver refusing the request. This is useful if we want to
try a different method of setting the VF MAC address before giving up
(Commit 86556e16 actually does this, setting the desired MAC address
to the "admin MAC in the PF, then detaching and reattaching the VF
netdev driver to force a reinit of the MAC address).

During testing of Bug 1442040 t was discovered that the ixgbe driver
returns EPERM in this situation, so this patch changes the exception
case for silent+non-terminal failure to account for this difference.

Completes resolution to: https://bugzilla.redhat.com/1415609 (RHEL 7.4)
                         https://bugzilla.redhat.com/1442040 (RHEL 7.3.z)

The current fallback stub for virNetDevSetCoalesce is inside an
earlier conditional block. This deals with the feature being
missing on older Linux platforms. We need a second fallback stub
though, outside the top level conditional, to ensure builds work
on Win32/FreeBSD platforms too.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch makes use of the virNetDevSetCoalesce() function to make
appropriate settings effective for devices that support them.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1414627Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

We are currently parsing only rx/frames/max because that's the only
value that makes sense for us.  The tun device just added support for
this one and the others are only supported by hardware devices which
we don't need to worry about as the only way we'd pass those to the
domain is using <hostdev/> or <interface type='hostdev'/>.  And in
those cases the guest can modify the settings itself.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

That function is able to configure coalesce settings for an interface,
similarly to 'ethtool -C'.  This function also updates back the
structure so that it contains actual data on the device (if the device
doesn't support some settings kernel might just return 0 and not set
whatever is not supported), so this way we'll have up-to-date
information in the live domain XML.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Reported by Rafał Wojciechowski <it@rafalwojciechowski.pl>.

Thread 1 (Thread 0x7f194b99d700 (LWP 5631)):
0  virNetDevGetifaddrsAddress (addr=0x7f194b99c7c0, ifname=0x7f193400e2b0 "ovirtmgmt") at util/virnetdevip.c:738
1  virNetDevIPAddrGet (ifname=0x7f193400e2b0 "ovirtmgmt", addr=addr@entry=0x7f194b99c7c0) at util/virnetdevip.c:795
2  0x00007f19467800d6 in networkGetNetworkAddress (netname=<optimized out>, netaddr=netaddr@entry=0x7f1924013f18) at network/bridge_driver.c:4780
3  0x00007f193e43a33c in qemuProcessGraphicsSetupNetworkAddress (listenAddr=0x7f19340f7650 "127.0.0.1", glisten=0x7f1924013f10) at qemu/qemu_process.c:4062
4  qemuProcessGraphicsSetupListen (vm=<optimized out>, graphics=0x7f1924014f10, cfg=0x7f1934119f00) at qemu/qemu_process.c:4133
5  qemuProcessSetupGraphics (flags=17, vm=0x7f19240155d0, driver=0x7f193411f1d0) at qemu/qemu_process.c:4196
6  qemuProcessPrepareDomain (conn=conn@entry=0x7f192c00ab50, driver=driver@entry=0x7f193411f1d0, vm=vm@entry=0x7f19240155d0, flags=flags@entry=17) at qemu/qemu_process.c:4969
7  0x00007f193e4417c0 in qemuProcessStart (conn=conn@entry=0x7f192c00ab50, driver=driver@entry=0x7f193411f1d0, vm=0x7f19240155d0,asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_START, migrateFrom=migrateFrom@entry=0x0, migrateFd=migrateFd@entry=-1, migratePath=migratePath@entry=0x0,snapshot=snapshot@entry=0x0, vmop=vmop@entry=VIR_NETDEV_VPORT_PROFILE_OP_CREATE, flags=17, flags@entry=1) at qemu/qemu_process.c:5553

Man page for getifaddrs also states that the "ifa_addr" may contain
a null pointer which happens if there is an existing network interface
on the host without IP address.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

libvirtd can spawn threads/tasks when creating new domains for
some hypervisors such as Xen's libxl driver, quickly reaching
the cgroups pids controller default TasksMax setting of 512. When
the limit is reached, attempting to create additional domains
results in an error from the cgroups pids controller, e.g.

kernel: [71282.213347] cgroup: fork rejected by pids controller in
/system.slice/libvirtd.service

Depending on domain type and configuration, anywhere from 4-7
threads/tasks may be created by libxl when starting a domain.
In order to support 4096 domains, similar to commit 27cd7635,
increase the TasksMax setting in libvirtd.service to
4096 * 8 = 32768 tasks.

In the RPC client event loop code, if poll() returns only a POLLHUP
or POLLERR status, then we end up reporting a bogus error message:

  error: failed to connect to the hypervisor
  error: An error occurred, but the cause is unknown

We do actually report an error, but we virNetClientMarkClose method
has already captured the error status before we report it, so the
real error gets thrown away. The key fix is to report the error
before calling virNetClientMarkClose(). In changing this, we also
split out reporting of POLLHUP vs POLLERR to make any future bugs
easier to diagnose.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In the vcpu hotplug code if exit from the monitor failed we would still
attempt to save the status XML. When the daemon is terminated the
monitor socket is closed. In such case, the written status XML would not
contain the monitor path and thus be invalid.

Avoid this issue by only saving status XML on success of the monitor
command.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1439452

The function is used only in the hotplug module.