In commit 63f0f45f the following
mechanical change was made:

         if (!state) {
-            qemu_aio_wait();
+            aio_poll(state->s->aio_context, true);
         }

The new code now checks if state is NULL and then dereferences it
('state->s') which is obviously incorrect.

This commit replaces state->s->aio_context with
bdrv_get_aio_context(bs), fixing this problem.  The two other hunks
are concerned with getting the BlockDriverState pointer bs to where it
is needed.

The original bug causes a segfault when using libguestfs to access a
VMware vCenter Server and doing any kind of complex read-heavy
operations.  With this commit the segfault goes away.
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NBenoît Canet <benoit.canet@nodalink.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

In order to access VMware ESX efficiently, we need to send a session
cookie.  This patch is very simple and just allows you to send that
session cookie.  It punts on the question of how you get the session
cookie in the first place, but in practice you can just run a `curl'
command against the server and extract the cookie that way.

To use it, add file.cookie to the curl URL.  For example:

$ qemu-img info 'json: {
    "file.driver":"https",
    "file.url":"https://vcenter/folder/Windows%202003/Windows%202003-flat.vmdk?dcPath=Datacenter&dsName=datastore1",
    "file.sslverify":"off",
    "file.cookie":"vmware_soap_session=\"52a01262-bf93-ccce-d379-8dabb3e55560\""}'
image: [...]
file format: raw
virtual size: 8.0G (8589934592 bytes)
disk size: unavailable
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The curl hardcoded timeout (5 seconds) sometimes is not long
enough depending on the remote server configuration and network
traffic. The user should be able to set how much long he is
willing to wait for the connection.

Adding a new option to set this timeout gives the user this
flexibility. The previous default timeout of 5 seconds will be
used if this option is not present.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Reviewed-by: NBenoit Canet <benoit.canet@nodalink.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Some code in the block layer makes potentially huge allocations. Failure
is not completely unexpected there, so avoid aborting qemu and handle
out-of-memory situations gracefully.

This patch addresses the allocations in the curl block driver.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NBenoit Canet <benoit@irqsave.net>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The curl block driver uses fd handlers, timers, and BHs.  The fd
handlers and timers are managed on behalf of libcurl, which controls
them using callback functions that the block driver implements.

The simplest way to implement .bdrv_detach/attach_aio_context() is to
clean up libcurl in the old event loop and initialize it again in the
new event loop.  We do not need to keep track of anything since there
are no pending requests when the AioContext is changed.

Also make sure to use aio_set_fd_handler() instead of
qemu_aio_set_fd_handler() and aio_bh_new() instead of qemu_bh_new() so
the current AioContext is passed in.

Cc: Alexander Graf <agraf@suse.de>
Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>

This allows qemu to use images over https with a self-signed certificate. It
defaults to verifying the certificate.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The block layer now supports a generic json syntax for passing option parameters
explicitly, making parsing of options from the url redundant.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

When receiving a new aio read request, we first look for an existing
transaction whose range will cover the read request by the time it
completes. However, we weren't checking that the existing transaction
was still active. If it had timed out, we were adding the request to a
transaction which would never complete and had already been cancelled,
resulting in a hang.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

According to the documentation, the correct way to ensure all
informationals have been returned by curl_multi_info_read is to loop
until it returns NULL.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

curl_multi_socket_all is a deprecated catch-all which checks for
activities on all open curl sockets. We have enough information from
the event loop to check only the sockets with activity. This change
removes use of curl_multi_socket_all in favour of
curl_multi_socket_action called with the relevant handle.

At the same time, it also ensures that the driver only checks for
completion of read operations after reading from a socket, rather than
both reading and writing.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Remove calls to curl_multi_do where the relevant handles are already
registered to the event loop.

Ensure that we kick off socket handling with CURL_SOCKET_TIMEOUT after
adding a new handle.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The driver will not start more than a fixed number of curl sessions.
If it needs more, it must wait for the completion of an existing one.
The driver was sleeping, which will prevent the main loop from
running, and therefore the event it's waiting on. It was also directly
calling its internal handler rather than waiting on existing
registered handlers to be called from the main loop.

This change causes it simply to wait for a period of time whilst
allowing the main loop to execute.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

A curl write callback is supposed to return the number of bytes it
handled.  curl_read_cb would have erroneously reported it had handled
all bytes in the event that the internal curl state was invalid.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This isn't any of the usually acceptable uses of goto.
Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NMatthew Booth <mbooth@redhat.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NMaria Kustova <maria.k@catit.be>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

curl_read_cb is callback function for libcurl when data arrives. The
data size passed in here is not guaranteed to be within the range of
request we submitted, so we may overflow the guest IO buffer. Check the
real size we have before memcpy to buffer to avoid overflow.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

error_is_set(&var) is the same as var != NULL, but it takes
whole-program analysis to figure that out.  Unnecessarily hard for
optimizers, static checkers, and human readers.  Dumb it down to
obvious.

Gets rid of several dozen Coverity false positives.

Note that the obvious form is already used in many places.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

libcurl versions 7.16.0 and later have a timer callback interface which
must be implemented in order for libcurl to make forward progress (it
will sometimes rely on being called back on the timeout if there are
no file descriptors registered). Implement the callback, and use a
QEMU AIO timer to ensure we prod libcurl again when it asks us to.

Based on Peter's original patch plus my fix to add curl_multi_timeout_do.
Should compile just fine even on older versions of libcurl.

I also tried copy-on-read and streaming:

    $ ./qemu-img create -f qcow2 -o \
         backing_file=http://download.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso \
         foo.qcow2 1G
    $ x86_64-softmmu/qemu-system-x86_64 \
         -drive if=none,file=foo.qcow2,copy-on-read=on,id=cd \
         -device ide-cd,drive=cd --enable-kvm -m 1024

Direct http usage is probably too slow, but with copy-on-read ultimately
the image does boot!

After some time, streaming gets canceled by an EIO, which needs further
investigation.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This is a boiler-plate _nofail variant of qemu_opts_create. Remove and
use error_abort in call sites.

null/0 arguments needs to be added for the id and fail_if_exists fields
in affected callsites due to argument inconsistency between the normal and
no_fail variants.
Signed-off-by: NPeter Crosthwaite <peter.crosthwaite@xilinx.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Add an Error ** parameter to BlockDriver.bdrv_open and
BlockDriver.bdrv_file_open to allow more specific error messages.
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Drop error code path which cannot be taken since qemu_bh_new() does not
return NULL.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

The .io_flush() handler no longer exists and has no users.  Drop the
io_flush argument to aio_set_fd_handler() and related functions.

The AioFlushEventNotifierHandler and AioFlushHandler typedefs are no
longer used and are dropped too.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

.io_flush() is no longer called so drop curl_aio_flush().  The acb[]
array that the function checks is still used in other parts of
block/curl.c.  Therefore we cannot remove acb[], it is needed.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

CURL driver requests partial data from server on guest IO req. For HTTP
and HTTPS, it uses "Range: ***" in requests, and this will not work if
server not accepting range. This patch does this check when open.

 * Removed curl_size_cb, which is not used: On one hand it's registered to
   libcurl as CURLOPT_WRITEFUNCTION, instead of CURLOPT_HEADERFUNCTION,
   which will get called with *data*, not *header*. On the other hand the
   s->len is assigned unconditionally later.

   In this gone function, the sscanf for "Content-Length: %zd", on
   (void *)ptr, which is not guaranteed to be zero-terminated, is
   potentially a security bug. So this patch fixes it as a side-effect. The
   bug is reported as: https://bugs.launchpad.net/qemu/+bug/1188943
   (Note the bug is marked "private" so you might not be able to see it)

 * Introduced curl_header_cb, which is used to parse header and mark the
   server as accepting range if "Accept-Ranges: bytes" line is seen from
   response header. If protocol is HTTP or HTTPS, but server response has
   no not this support, refuse to open this URL.

Note that python builtin module SimpleHTTPServer is an example of not
supporting range, if you need to test this driver, get a better server
or use internet URLs.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

(Found by Kamil Dudka)
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Cc: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Trivial patch to remove odd whitespace.
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

It is unused now in all block drivers.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

As a bonus, going through the QemuOpts QEMU_OPT_SIZE parser for the
readahead option gives us proper error reporting that the previous use
of atoi() lacked.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The new parameter is unused yet.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, ...) interface was
introduced in libcurl 7.19.4.  Therefore we cannot protect against
CVE-2013-0249 when linking against an older libcurl.

This fixes the build failure introduced by
fb6d1bbd.
Reported-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Tested-by: NAndreas Färber <andreas.faeber@web.de>
Message-id: 1360743934-8337-1-git-send-email-stefanha@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

There is a buffer overflow in libcurl POP3/SMTP/IMAP.  The workaround is
simple: disable extra protocols so that they cannot be exploited.  Full
details here:

  http://curl.haxx.se/docs/adv_20130206.html

QEMU only cares about HTTP, HTTPS, FTP, FTPS, and TFTP.  I have tested
that this fix prevents the exploit on my host with
libcurl-7.27.0-5.fc18.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Now that AIOPool no longer keeps a freelist, it isn't really a "pool"
anymore.  Rename it to AIOCBInfo and make it const since it no longer
needs to be modified.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Report from smatch:
block/curl.c:546 curl_close(21) info: redundant null check on s->url calling free()

The check was redundant, and free was also wrong because the memory
was allocated using g_strdup.
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Similar to
 qemu_iovec_memset(QEMUIOVector *qiov, size_t offset,
                   int c, size_t bytes);
the new prototype is:
 qemu_iovec_from_buf(QEMUIOVector *qiov, size_t offset,
                     const void *buf, size_t bytes);

The processing starts at offset bytes within qiov.

This way, we may copy a bounce buffer directly to
a middle of qiov.

This is exactly the same function as iov_from_buf() from
iov.c, so use the existing implementation and rename it
to qemu_iovec_from_buf() to be shorter and to match the
utility function.

As with utility implementation, we now assert that the
offset is inside actual iovec.  Nothing changed for
current callers, because `offset' parameter is new.

While at it, stop using "bounce-qiov" in block/qcow2.c
and copy decrypted data directly from cluster_data
instead of recreating a temp qiov for doing that.
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>