A single PCI device may have multiple netdevs associated with it. Each
of those netdevs will have a different phys_port_id entry in
sysfs. This patch modifies virPCIGetNetName() to allow selecting one
of the potential many netdevs in two different ways:

1) by setting the "idx" argument, the caller can select the 1st (0),
2nd (1), etc. netdev from the PCI device's net subdirectory.

2) If the physPortID arg is set (to a null-terminated string) then
virPCIGetNetName() returns the netdev that has that phys_port_id in
the sysfs file of the same name in the netdev's directory.

The first arg isn't modified in the function, so it should be const.

On Linux each network device *can* (but not necessarily *does*) have
an attribute called phys_port_id which can be read from the file of
that name in the netdev's sysfs directory. The examples I've seen have
been a many-digit hexadecimal number (as an ASCII string).

This value can be useful when a single PCI device is associated with
multiple netdevs (e.g a dual port Mellanox SR-IOV NIC - this card has
a single PCI Physical Function (PF), and that PF has two netdevs
associated with it (the "net" subdirectory of the PF in sysfs has two
links rather than the usual single link to a netdev directory). Each
of the PF netdevs has a different phys_port_id. The Virtual Functions
(VF) are similar - the PF (a PCI device) has "n" VFs (also each of
these is a PCI device), each VF has two netdevs, and each of the VF
netdevs points back to the VF PCI device (with the "device" entry in
its sysfs directory) as well as having a phys_port_id matching the PF
netdev it is associated with.

virNetDevGetPhysPortID() simply attempts to read the phys_port_id for
the given netdev and return it to the caller. If this particular
netdev driver doesn't support phys_port_id, it returns NULL (*not* a
NULL-terminated string, but a NULL pointer) but still counts it as a
success.

Rewrite virConfReadMem to take a null-terminated string.
All the callers were calling strlen on it anyway.

Export the function from the util module so that dynamic linking can
override it.

The functions declared in virhash.h return bool, but stdbool.h was not
included.

We are given a string in @machinename, we never allocate it, just
merely use it for reading. We should not free it otherwise it
leads to double free:

==32191== Thread 17:
==32191== Invalid free() / delete / delete[] / realloc()
==32191==    at 0x4C2D1A0: free (vg_replace_malloc.c:530)
==32191==    by 0x54BBB84: virFree (viralloc.c:582)
==32191==    by 0x2BC04499: qemuProcessStop (qemu_process.c:6313)
==32191==    by 0x2BC500FF: processMonitorEOFEvent (qemu_driver.c:4724)
==32191==    by 0x2BC502FC: qemuProcessEventHandler (qemu_driver.c:4769)
==32191==    by 0x5550640: virThreadPoolWorker (virthreadpool.c:167)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)
==32191==    by 0x928DE3C: clone (in /lib64/libc-2.23.so)
==32191==  Address 0x31893d70 is 0 bytes inside a block of size 1,100 free'd
==32191==    at 0x4C2D1A0: free (vg_replace_malloc.c:530)
==32191==    by 0x54BBB84: virFree (viralloc.c:582)
==32191==    by 0x54C1936: virCgroupValidateMachineGroup (vircgroup.c:343)
==32191==    by 0x54C4B29: virCgroupNewDetectMachine (vircgroup.c:1550)
==32191==    by 0x2BBDDA29: qemuConnectCgroup (qemu_cgroup.c:972)
==32191==    by 0x2BC05DA7: qemuProcessReconnect (qemu_process.c:6822)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)
==32191==    by 0x928DE3C: clone (in /lib64/libc-2.23.so)
==32191==  Block was alloc'd at
==32191==    at 0x4C2BE80: malloc (vg_replace_malloc.c:298)
==32191==    by 0x4C2E35F: realloc (vg_replace_malloc.c:785)
==32191==    by 0x54BB492: virReallocN (viralloc.c:245)
==32191==    by 0x54BEDF2: virBufferGrow (virbuffer.c:150)
==32191==    by 0x54BF3B9: virBufferVasprintf (virbuffer.c:408)
==32191==    by 0x54BF324: virBufferAsprintf (virbuffer.c:381)
==32191==    by 0x55BB271: virDomainGenerateMachineName (domain_conf.c:27078)
==32191==    by 0x2BBD5B8F: qemuDomainGetMachineName (qemu_domain.c:9595)
==32191==    by 0x2BBDD9B4: qemuConnectCgroup (qemu_cgroup.c:966)
==32191==    by 0x2BC05DA7: qemuProcessReconnect (qemu_process.c:6822)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)

Moreover, make the @machinename 'const char *' to mark it
explicitly that we are not changing the passed string.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Make it less confusing by naming the field which refers to the storage
object as 'nodestorage'.
Reviewed-by: NEric Blake <eblake@redhat.com>

Split out the worker loop into a separate function and export it.
Reviewed-by: NEric Blake <eblake@redhat.com>

The new virFileCache will nicely handle the caching logic for any data
that we would like to cache.  For each type of data we will just need
to implement few handlers that will take care of creating, validating,
loading and saving the cached data.

The cached data must be an instance of virObject.

Currently we cache QEMU capabilities which will start using
virFileCache.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

It is more related to a domain as we might use it even when there is
no systemd and it does not use any dbus/systemd functions.  In order
not to use code from conf/ in util/ pass machineName in cgroups code
as a parameter.  That also fixes a leak of machineName in the lxc
driver and cleans up and de-duplicates some code.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

This reverts commit 328bd244.

As it turns out, this is not portable and very Linux & glibc
specific. Worse, this may lead to not starving writers on Linux
but everywhere else. Revert this and if the starvation occurs
resolve it.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel P. Berrange <berrange@redhat.com>

Up until now we only had virObjectLockable which uses mutexes for
mutually excluding each other in critical section. Well, this is
not enough. Future work will require RW locks so we might as well
have virObjectRWLockable which is introduced here.

Moreover, polymorphism is introduced to our code for the first
time. Yay! More specifically, virObjectLock will grab a write
lock, virObjectLockRead will grab a read lock then (what a
surprise right?). This has great advantage that an object can be
made derived from virObjectRWLockable in a single line and still
continue functioning properly (mutexes can be viewed as grabbing
write locks only). Then just those critical sections that can
grab a read lock need fixing. Therefore the resulting change is
going to be way smaller.

In order to avoid writer starvation, the object initializes RW
lock that prefers writers.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

We already have virRWLockInit. But this uses pthread defaults
which prefer reader to initialize the RW lock. This may lead to
writer starvation. Therefore we need to have the counterpart that
prefers writers. Now, according to the
pthread_rwlockattr_setkind_np() man page setting
PTHREAD_RWLOCK_PREFER_WRITER_NP attribute is no-op. Therefore we
need to use PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP
attribute. So much for good enum value names.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Currently, @port is type of string. Well, that's overkill and
waste of memory. Port is always an integer. Use it as such.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Similarly to when parsing XML we need to fill in default ports for the
backing chain. This was missed in commit 5bda8354

Negative ports don't make sense so use a unsigned integer.

Move it to virstring.c and improve it to parse and validate ports. New
name is virStringParsePort.

Since virBitmapSubtract is unused modify it to perform bitmap
intersection.

A new function virNetDevOpenvswitchUpdateVlan has been created to instruct
OVS of the changes. qemuDomainChangeNet has been modified to handle the
update of the VLAN configuration for a running guest and rely on
virNetDevOpenvswitchUpdateVlan to do the actual update if needed.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This piece of code is going to be reused. So move it out to a
separate function.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

While searching for an element using a function it may be
desirable to know the element key for future operation.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

The purpose of this function is to tell if the current position
in given FD is in data section or a hole and how much bytes there
is remaining until the end of the section. This is achieved by
couple of lseeks(). The most important part is that we reposition
the FD back, so that the position is unchanged from the caller
POV. And until now the final lseek() back to the original
position was done with no check for errors. And I was convinced
that that's okay since nothing can go wrong. However, review
feedback from a related series persuaded me, that it's better to
be safe than sorry. Therefore, lets check if the final lseek()
succeeded and if it doesn't report an error.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

Fill them in right away rather than having to figure out at runtime
whether they are necessary or not.

virStorageSourceNetworkDefaultPort does not need to be exported any
more.

Our documentation provides them, so the helper should return them.

Make the stuff hardcoded in qemu a global helper so that other parts of
the code can determine the default port too.

This reverts commit e4b980c8.

When a binary links against a .a archive (as opposed to a shared library),
any symbols which are marked as 'weak' get silently dropped. As a result
when the binary later runs, those 'weak' functions have an address of
0x0 and thus crash when run.

This happened with virtlogd and virtlockd because they don't link to
libvirt.so, but instead just libvirt_util.a and libvirt_rpc.a. The
virRandomBits symbols was weak and so left out of the virtlogd &
virtlockd binaries, despite being required by virHashTable functions.

Various other binaries like libvirt_lxc, libvirt_iohelper, etc also
link directly to .a files instead of libvirt.so, so are potentially
at risk of dropping symbols leading to a later runtime crash.

This is normal linker behaviour because a weak symbol is not treated
as undefined, so nothing forces it to be pulled in from the .a You
have to force the linker to pull in weak symbols using -u$SYMNAME
which is not a practical approach.

This risk is silent bad linkage that affects runtime behaviour is
not acceptable for a fix that was merely trying to fix the test
suite. So stop using __weak__ again.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Currently the scan of the /proc/mounts file used to find cgroup mount
points doesn't take into account that mount points may hidden by other
mount points. For, example in certain Kubernetes environments the
/proc/mounts contains the following lines:

  cgroup /sys/fs/cgroup/net_prio,net_cls cgroup ...
  tmpfs /sys/fs/cgroup tmpfs ...
  cgroup /sys/fs/cgroup/net_cls,net_prio cgroup ...

In this particular environment the first mount point is hidden by the
second one. The correct mount point is the third one, but libvirt will
never process it because it only checks the first mount point for each
controller (net_cls in this case). So libvirt will try to use the first
mount point, which doesn't actually exist, and the complete detection
process will fail.

To avoid that issue this patch changes the virCgroupDetectMountsFromFile
function so that when there are duplicates it takes the information from
the last line in /proc/mounts. This requires removing the previous
explicit condition to skip duplicates, and adding code to free the
memory used by the processing of duplicated lines.

Related-To: https://bugzilla.redhat.com/1468214
Related-To: https://github.com/kubevirt/libvirt/issues/4Signed-off-by: NJuan Hernandez <jhernand@redhat.com>

Currently all mockable functions are annotated with the 'noinline'
attribute. This is insufficient to guarantee that a function can
be reliably mocked with an LD_PRELOAD. The C language spec allows
the compiler to assume there is only a single implementation of
each function. It can thus do things like propagating constant
return values into the caller at compile time, or creating
multiple specialized copies of the function body each optimized
for a different caller. To prevent these optimizations we must
also set the 'noclone' and 'weak' attributes.

This fixes the test suite when libvirt.so is built with CLang
with optimization enabled.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The HOST_NAME_MAX, INET_ADDRSTRLEN and VIR_LOOPBACK_IPV4_ADDR
constants are only used by a handful of files, so are better
kept in virsocketaddr.h or the source file that uses them.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Now that the JSON deflattener is working sanely we can always attempt
the deflattening so that we can then parse the tree as expected.

If a value of the first level object contains more objects needing
deflattening which would be wrapped in an actual object the function
would not recurse into them.

By this simple addition we can fully deflatten the objects.

As it turns out sometimes users pass in an arbitrarily nested structure
e.g. for the qemu backing chains JSON pseudo protocol. This new
implementation deflattens now a single object fully even with nested
keys.

Additionally it's not necessary now to stick with the "file." prefix for
the properties.

Currently the function would deflatten the object by dropping the 'file'
prefix from the attributes. This does not really scale well or adhere to
the documentation.

Until we refactor the worker to properly deflatten everything we at
least simulate it by adding the "file" wrapper object back.

The code will become more universal so it makes more sense for it to
live with the rest of the JSON functions.

Allows testing whether a virJSONValue is an object.

virCommand is a version of virExec that doesn't fork, however it is
just calling execve and doesn't honors setting uid/gid and pwd.

This commit extrac those pieces from virExec() to a virExecCommon()
function that is called from both virExec() and virCommandExec().

Problem with our error reporting is that the error object is a
thread local variable. That means if there's an error reported
within the I/O thread it gets logged and everything, but later
when the event loop aborts the stream it doesn't see the original
error. So we are left with some generic error. We can do better
if we copy the error message between the threads.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>

When the I/O thread quits (e.g. due to an I/O error, lseek()
error, whatever), any subsequent virFDStream API should return
error too. Moreover, when invoking stream event callback, we must
set the VIR_STREAM_EVENT_ERROR flag so that the callback knows
something bad happened.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJohn Ferlan <jferlan@redhat.com>