When using multi-pass authentication methods, the curl library may
need to rewind the read buffers (depending on how much already has
been fed to the server) used for providing data to HTTP PUT, POST or
PROPFIND, and in order to allow the library to do so, we need to tell
it how by providing either an ioctl callback or a seek callback.

This patch adds an ioctl callback, which should be usable on older
curl versions (since 7.12.3) than the seek callback (introduced in
curl 7.18.0).

Some HTTP servers (such as Apache) give an 401 error reply immediately
after receiving the headers (so no data has been read from the read
buffers, and thus no rewinding is needed), but other servers (such
as Lighttpd) only replies after the whole request has been sent and
all data has been read from the read buffers, making rewinding necessary.
Signed-off-by: NMartin Storsjo <martin@martin.st>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Besides, we have already called easy_setopt with the option before coming
to this function if it was available, so there is no need to repeat it
here.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Curl is designed not to ask for password when only username is given in
the URL, but has a way for application to feed a (username, password) pair
to it.  With this patch, you do not have to keep your password in
plaintext in your $HOME/.netrc file when talking with a password protected
URL with http://<username>@<host>/path/to/repository.git/ syntax.

The code handles only the http-walker side, not the push side.  At least,
not yet.  But interested parties can add support for it.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We honor the command line options, environment variables, variables in
repository configuration file, variables in user's global configuration
file, variables in the system configuration file, and then finally use
built-in default.  To implement this semantics, the code should:

 - start from built-in default values;

 - call git_config() with the configuration parser callback, which
   implements "later definition overrides earlier ones" logic
   (git_config() reads the system's, user's and then repository's
   configuration file in this order);

 - override the result from the above with environment variables if set;

 - override the result from the above with command line options.

The initialization code http_init() for http transfer got this wrong, and
implemented a "first one wins, ignoring the later ones" in http_options(),
to compensate this mistake, read environment variables before calling
git_config().  This is all wrong.

As a second class citizen, the http codepath hasn't been audited as
closely as other parts of the system, but we should try to bring sanity to
it, before inviting contributors to improve on it.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In addition, ''quote_ref_url'' inserts a slash between the base URL and
remote ref path only if needed. Previously, this insertion wasn't
contingent on the lack of a separating slash.
Signed-off-by: NTay Ray Chuan <rctay89@gmail.com>
Acked-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NMiklos Vajna <vmiklos@frugalware.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Some places use the standard malloc/strdup without checking if the
allocation was successful; they should use xmalloc/xstrdup that
check the memory allocation result.
Signed-off-by: NDotan Barak <dotanba@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Originally from Mike Hommey; earlier we were disabling SSL_VERIFYPEER
but SSL_VERIFYHOST was in effect even when the user asked not to with
the environment variable.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NBrian Hetro <whee@smaertness.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

After master.k.org upgrade, I started seeing these warning messages:

    transport.c: In function 'get_refs_via_curl':
    transport.c:458: error: call to '_curl_easy_setopt_err_write_callback' declared with attribute warning: curl_easy_setopt expects a curl_write_callback argument for this option

It appears that the curl header wants to enforce the function signature
for callback function given to curl_easy_setopt() to be compatible with
that of (*curl_write_callback) or fwrite.  This patch seems to work the
issue around.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In c13b2633, http_fetch_ref got "refs/" included in the ref passed to it,
which, incidentally, makes the allocation in quote_ref_url too big, now.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

git_config() only had a function parameter, but no callback data
parameter.  This assumes that all callback functions only modify
global variables.

With this patch, every callback gets a void * parameter, and it is hoped
that this will help the libification effort.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This makes a struct ref able to represent a symref, and makes http.c
able to recognize one, and makes transport.c look for "HEAD" as a ref
in the list, and makes it dereference symrefs for the resulting ref,
if any.
Signed-off-by: NDaniel Barkalow <barkalow@iabervon.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This simplifies a few things, makes a few things slightly more
complicated, but, more importantly, allows that, when struct ref can
represent a symref, http_fetch_ref() can return one.

Incidentally makes the string that http_fetch_ref() gets include "refs/"
(if appropriate), because that's how the name field of struct ref works.
As far as I can tell, the usage in walker:interpret_target() wouldn't have
worked previously, if it ever would have been used, which it wouldn't
(since the fetch process uses the hash instead of the name of the ref
there).
Signed-off-by: NDaniel Barkalow <barkalow@iabervon.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For some reason, http_cleanup was running all active slots, which could
lead in situations where a freed slot would be accessed in
fill_active_slots. OTOH, we are cleaning up, which means the caller
doesn't care about pending requests. Just forget about them instead
or running them.
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In transport.c, proxy setting (the one from the remote conf) was set through
curl_easy_setopt() call, while http.c already does the same with the
http.proxy setting. We now just use this infrastructure instead, and make
http_init() now take the struct remote as argument so that it can take the
http_proxy setting from there, and any other property that would be added
later.

At the same time, we make get_http_walker() take a struct remote argument
too, and pass it to http_init(), which makes remote defined proxy be used
for more than get_refs_via_curl().

We leave out http-fetch and http-push, which don't use remotes for the
moment, purposefully.
Signed-off-by: NMike Hommey <mh@glandium.org>
Acked-by: NDaniel Barkalow <barkalow@iabervon.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

http.sslcert and friends expect a string value
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Make the necessary changes to be ok with their difference, and rename the
function http_fetch_ref.
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Also, replace whitespaces with tabs in some places
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Setting CURLOPT_HTTPHEADER doesn't add HTTP headers, but replaces whatever
set of headers was configured before, so setting to NULL doesn't have any
magic meaning, and is pretty much useless when setting to another list
right after.
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Quite some variables defined as extern in http.h are only used in http.c,
and some others, only defined in http.c, were not static.
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The http_proxy / HTTPS_PROXY variables used by curl to control
proxying may not be suitable for git.  Allow the user to override them
in the configuration file.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Junio and I both noticed that the new builtin-fetch was segfaulting
immediately on http/https/ftp style URLs (those that went through
libcurl and the commit walker).  Although the builtin-fetch changes
in this area were really just minor refactorings there was one major
change made: we invoked http_init(), http_cleanup() then http_init()
again in the same process.

When we call curl_easy_cleanup() on each active_request_slot we
are telling libcurl we did not want that buffer to be used again.
Unfortunately we did not also deallocate the active_request_slot
itself nor did we NULL out active_queue_head.  This lead us to
attempt to reuse these cleaned up libcurl handles when we later tried
to invoke http_init() a second time to reactivate the curl library.
The next file get operation then immediately segfaulted on most
versions of libcurl.

Properly freeing our own buffers and clearing the list causes us to
reinitialize the curl buffers again if/when we need to use libcurl
from within this same process.
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This eliminates the last function provided by the code using http.h as
a global symbol, so it should be possible to have multiple programs
using http.h in the same executable, and it also adds an argument to
that callback, so that info can be passed into the callback without
being global.
Signed-off-by: NDaniel Barkalow <barkalow@iabervon.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This removes all of the boilerplate and http-internal stuff from
fill_active_slots() and makes it easy to turn into a callback.
Signed-off-by: NDaniel Barkalow <barkalow@iabervon.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This uses "git-apply --whitespace=strip" to fix whitespace errors that have
crept in to our source files over time.  There are a few files that need
to have trailing whitespaces (most notably, test vectors).  The results
still passes the test, and build result in Documentation/ area is unchanged.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Calling http_init after calling http_cleanup causes a segfault.  This
is due to the pragma_header curl_slist being freed but not being set
to NULL.  The subsequent call to http_init tries to setup the slist
again, but it now points to an invalid memory location.
Signed-off-by: NJulian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

If http.noEPSV config variable is defined and true, or if
GIT_CURL_FTP_NO_EPSV environment variable is defined, disable using
of EPSV ftp command (PASV will be used instead). This is helpful with
some "poor" ftp servers which does not support EPSV mode.
Signed-off-by: NSasha Khapyorsky <sashak@voltaire.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

ANSI C99 doesn't allow void-pointer arithmetic. This patch fixes this in
various ways. Usually the strategy that required the least changes was used.
Signed-off-by: NFlorian Forster <octo@verplant.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Fix broken build when USE_CURL_MULTI is not defined, as noted by Becky Bruce.

During cleanup, free header slist that was created during init, as noted
by Junio.
Signed-off-by: NNick Hengeveld <nickh@reactrix.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

If a curl handle is configured with special options, they may reference
information that is freed after the request is complete which can cause
a segfault if the curl handle is reused for a different type of request.

This patch resets these options to a safe state when a transfer slot is
assigned to a new request.
Signed-off-by: NNick Hengeveld <nickh@reactrix.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Useful for diagnostics/troubleshooting to know which client versions are
hitting your server.
Signed-off-by: NNick Hengeveld <nickh@reactrix.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Incorporate into http-push a fix related to accessing slot results after
the slot was reused, and fix a case in run_active_slot where a
finished slot wasn't detected if the slot was reused.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

In fetch_object, there's a call to release an object request if the
object mysteriously arrived, say in a pack.  Unfortunately, the fetch
attempt for this object might already be in progress, and we'll leak the
descriptor.  Instead, try to tidy away the request.
Signed-off-by: NMark Wooding <mdw@distorted.org.uk>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NMark Wooding <mdw@distorted.org.uk>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Add a way to store the results of an HTTP request when a slot finishes
so the results can be processed after the slot has been reused.
Signed-off-by: NNick Hengeveld <nickh@reactrix.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Move the static function get_curl_handle() around to make sure
its definition and declarations are seen by the compiler before
its first use.  Also remove an unused variable.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Move shared HTTP request functionality out of http-fetch and http-push,
and replace the two fwrite_buffer/fwrite_buffer_dynamic functions with
one fwrite_buffer function that does dynamic buffering.  Use slot
callbacks to process responses to fetch object transfer requests and
push transfer requests, and put all of http-push into an #ifdef check
for curl multi support.
Signed-off-by: NNick Hengeveld <nickh@reactrix.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>