GCC 4.6 complains about a local declaration shadowing a global symbol.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Commit e3ba4025 introduced srv->handles and VIR_RESIZE_N to allocate
@handles as necessary, but did not free the handles during when calling
virNetlinkEventServiceStop.

Commit 15a71e60 introduced the virNetlinkEventServiceStopAll function, and
the code in virNetlinkEventServiceStop is copied to this function, so just
call virNetlinkEventServiceStop instead.

https://bugzilla.redhat.com/show_bug.cgi?id=1452072Signed-off-by: NErik Skultety <eskultet@redhat.com>

Start discovering the mediated devices on the host system and format the
attributes for the mediated device into the XML. Compared to the parent
device which reports generic information about the abstract mediated
devices types, a child device only reports the type name it has been
instantiated from and the IOMMU group number, since that's device
specific compared to the rest of the info that can be gathered about
mediated devices at the moment.
This patch introduces both the formatting and parsing routines, updates
nodedev.rng schema, adding a testcase as well.

The resulting mdev child device XML:
<device>
  <name>mdev_4b20d080_1b54_4048_85b3_a6a62d165c01</name>
  <path>/sys/devices/.../4b20d080-1b54-4048-85b3-a6a62d165c01</path>
  <parent>pci_0000_06_00_0</parent>
  <driver>
    <name>vfio_mdev</name>
  </driver>
  <capability type='mdev'>
    <type id='vendor_supplied_type_id'/>
    <iommuGroup number='NUM'/>
  <capability/>
<device/>

https://bugzilla.redhat.com/show_bug.cgi?id=1452072Signed-off-by: NErik Skultety <eskultet@redhat.com>

The parent device needs to report the generic stuff about the supported
mediated devices types, like device API, available instances, type name,
etc. Therefore this patch introduces a new nested capability element of
type 'mdev_types' with the resulting XML of the following format:

<device>
  ...
  <capability type='pci'>
    ...
    <capability type='mdev_types'>
      <type id='vendor_supplied_id'>
        <name>optional_vendor_supplied_codename</name>
        <deviceAPI>vfio-pci</deviceAPI>
        <availableInstances>NUM</availableInstances>
      </type>
        ...
      <type>
        ...
      </type>
    </capability>
  </capability>
  ...
</device>

https://bugzilla.redhat.com/show_bug.cgi?id=1452072Signed-off-by: NErik Skultety <eskultet@redhat.com>

The reason for introducing two capabilities, one for the device itself
(cap 'mdev') and one for the parent device listing the available types
('mdev_types'), is that we should be able to do
'virsh nodedev-list --cap' not only for existing mdev devices but also
for devices that support creation of mdev devices, since one day libvirt
might be actually able to create the mdev devices in an automated way
(just like we do for NPIV/vHBA).

https://bugzilla.redhat.com/show_bug.cgi?id=1452072Signed-off-by: NErik Skultety <eskultet@redhat.com>

Since there's at least SRIOV and MDEV sub-capabilities to be parsed,
let's make the code more readable by splitting it to several logical
blocks.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Namely, this patch is about virMediatedDeviceGetIOMMUGroup{Dev,Num}
functions. There's no compelling reason why these functions should take
an object, on the contrary, having to create an object every time one
needs to query the IOMMU group number, discarding the object afterwards,
seems odd.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Commit 8e09663f "pci: recognize/report GEN4 (PCIe 4.0) card 16GT/s Link
speed" introduced another speed into enum, but mistakenly also altered
field width, so one bit of link width was included there.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Similarly to previous commit, implement sparse streams feature
for vol-upload. This is, however, slightly different approach,
because we must implement a function that will tell us whether
we are in a data section or in a hole. But there's no magic
hidden in here.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add a new --sparse switch that does nothing more than
enables the sparse streams feature for this command. Among with
the switch new helper function is introduced: virshStreamSkip().
This is the callback that is called whenever daemon sends us a
hole. In the callback we reflect the hole in underlying file by
seeking as many bytes as told.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

These flags to APIs will tell if caller wants to use sparse
stream for storage transfer. At the same time, it's safe to
enable them in storage driver frontend and rely on our backends
checking the flags. This way we can enable specific flags only on
some specific backends, e.g. enable
VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM for filesystem backend but
not iSCSI backend.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Now, not all APIs are going to support sparse streams. To some it
makes no sense at all, e.g. virDomainOpenConsole() or
virDomainOpenChannel(). To others, we will need a special flag to
indicate that client wants to enable sparse streams. Instead of
having to write RPC dispatchers by hand we can just annotate in
our .x files that a certain flag to certain RPC call enables this
feature. For instance:

     /**
      * @generate: both
      * @ReadStream: 1
      * @sparseflag: VIR_SPARSE_STREAM
      * @acl: storage_vol:data_read
      */
     REMOTE_PROC_DOMAIN_SOME_API = XXX,

Therefore, whenever client calls virDomainSomeAPI(..,
VIR_SPARSE_STREAM); daemon will mark that down and send stream
skips when possible.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Basically, what is needed here is to introduce new message type
for the messages passed between the event loop callbacks and the
worker thread that does all the I/O. The idea is that instead of
a queue of read buffers we will have a queue where "hole of size
X" messages appear. That way the event loop callbacks can just
check the head of the queue and see if the worker thread is in
data or a hole section and how long the section is.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Whenever client is able to receive some data from stream
daemonStreamHandleRead is called. But now the behaviour of this
function needs to be changed a bit. Previously it just read data
from underlying file (of chardev or whatever) and sent those
through the stream to client. This model will not work any longer
because it does not differentiate whether underlying file is in
data or hole section. Therefore, at the beginning of this
function add code that checks this situation and acts
accordingly.
So after the this, when wanting to send some data we always check
whether we are not in a hole and if so, skip it an inform client
about its size.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is fairly trivial now that we have everything in place.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Whenever server sends a client stream packet (either regular with
actual data or stream skip one) it is queued on @st->rx. So the
list is a mixture of both types of stream packets. So now that we
have all the helpers needed we can wire their processing up. But
since virNetClientStreamRecvPacket doesn't support
VIR_STREAM_RECV_STOP_AT_HOLE flag yet, let's turn all received
skips into zeroes repeating requested times.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This function will fetch previously processed stream holes and
return their sum.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Now that we have RPC wrappers over VIR_NET_STREAM_HOLE we can
start wiring them up. This commit wires up situation when a
client wants to send a hole to daemon.

To keep stream offsets synchronous, upon successful call on the
daemon skip the same hole in local part of the stream.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is a function that handles an incoming STREAM_HOLE packet.
Even though it is not wired up yet, it will be soon. At the
beginning do couple of checks whether server plays nicely and
sent us a STREAM_HOLE packed only after we've enabled sparse
streams. Then decodes the message payload to see how big the hole
is and stores it in passed @length argument.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Basically, whenever the new type of stream packet arrives to the
daemon call this function that decodes it and calls
virStreamSendHole(). Otherwise a regular data stream packet has
arrived and therefore continue its processing.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

While the previous commit implemented a helper for sending a
STREAM_HOLE packet for daemon, this is a client's counterpart.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is just a helper function that takes in a length value,
encodes it into XDR and sends to client.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Ideally, this would be generated, but to achieve that
corresponding XDR definitions needed to go into a different .x
file. But they belong just to the one that they are right now.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is a special type of stream packet, that is bidirectional
and contains information regarding how many bytes each side will
be skipping in the stream.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is going to be RPC representation for virStreamSendHole.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add a new argument to daemonCreateClientStream in order to allow for
future expansion to mark that a specific stream can be used to skip
data, such as the case with sparsely populated files. The new flag will
be the eventual decision point between client/server to decide whether
both ends can support and want to use sparse streams.

A new bool 'allowSkip' is added to both _virNetClientStream and
daemonClientStream in order to perform the tracking.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add a virStreamPtr pointer to the _virNetClientStream
in order to reverse track the parent stream.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is just an internal API, that calls corresponding function
in stream driver. This function will set @data = 1 if the
underlying file is in data section, or @data = 0 if it is in a
hole. At any rate, @length is set to number of bytes remaining in
the section the file currently is.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is just a wrapper over new function that have been just
introduced: virStreamSendHole() . It's very similar to
virStreamSendAll() except it handles sparse streams well.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is just a wrapper over new functions that have been just
introduced: virStreamRecvFlags(), virStreamRecvHole(). It's very
similar to virStreamRecvAll() except it handles sparse streams
well.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add a new flag to virStreamRecvFlags in order to handle being able to
stop reading from the stream so that the consumer can generate a "hole"
in stream target. Generation of a hole replaces the need to receive and
handle a sequence of zero bytes for sparse stream targets.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This function is basically a counterpart for virStreamSendHole().
If one side of a stream called virStreamSendHole() the other
should call virStreamRecvHole() to get the size of the hole.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This API is used to tell the other side of the stream to skip
some bytes in the stream. This can be used to create a sparse
file on the receiving side of a stream.

It takes @length argument, which says how big the hole is. This
skipping is done from the current point of stream. Since our
streams are not rewindable like regular files, we don't need
@whence argument like seek(2) has.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

There are three virStreamDriver's currently supported:

 * virFDStream
 * remote driver
 * ESX driver

For now, backend virStreamRecvFlags support for only remote driver and
ESX driver is sufficient. Future patches will update virFDStream.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This patch is adding the virStreamRecvFlags as a variant to the
virStreamRecv function in order to allow for future expansion of
functionality for processing sparse streams using a @flags
argument.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This function takes a FD and determines whether the current
position is in data section or in a hole. In addition to that,
it also determines how much bytes are there remaining till the
current section ends.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>