Add tests to check the behavior of fetching from a repository which
changes between rounds of negotiation (for example, when different
servers in a load-balancing agreement participate in the same stateless
RPC negotiation). This forms a baseline of comparison to the ref-in-want
functionality (which will be introduced to the client in subsequent
commits), and ensures that subsequent commits do not change existing
behavior.

As part of this effort, a mechanism to substitute strings in a single
HTTP response is added.
Signed-off-by: NBrandon Williams <bmwill@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This allows us to use common test infrastructure and parallelize
the tests.  For now, GIT_SVN_TEST_HTTPD=true needs to be set to
enable the SVN HTTP tests because we reuse the same test cases
for both file:// and http:// SVN repositories.  SVN_HTTPD_PORT
is no longer honored.

Tested under Apache 2.2 and 2.4 on Debian 7.x (wheezy) and
8.x (jessie), respectively.

Cc: Clemens Buchacher <drizzd@aon.at>
Cc: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: NEric Wong <e@80x24.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Failure to bring up httpd for testing is not considered an error, so the
trash directory, which contains this error.log file, is removed and we
don't know what made httpd fail to start. Improve the situation a bit,
print error.log but only in verbose mode.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
	perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg'  "${_f}"
done

and then carefully proof-read.
Signed-off-by: NElia Pinto <gitter.spiros@gmail.com>
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When built with NO_EXPAT=NoThanks, we will not have a working http-push
over webdav.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

If we built git without curl, we can't actually test against
an http server. In fact, all of the test scripts which
include lib-httpd.sh already perform this check, with one
exception: t5540. For those scripts, this is a noop, and for
t5540, this is a bugfix (it used to fail when built with
NO_CURL, though it could go unnoticed if you had a stale
git-remote-https in your build directory).
Noticed-by: NJunio C Hamano <junio@pobox.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Apache removes GIT_TRACE from the environment before running
git-http-backend. This can make it hard to debug the server
side of an http session. Let's let it through.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The SANITY prerequisite is really about whether the
filesystem will respect the permissions we set, and being
root is only one part of that. But the httpd tests really
just care about not being root, as they are trying to avoid
weirdness in apache (see a1a30111 for details).

Let's switch out SANITY for a new NOT_ROOT prerequisite,
which will let us tweak SANITY more freely.

We implement NOT_ROOT by checking `id -u`, which is in POSIX
and seems to be available even on MSYS.  Note that we cannot
just call this "ROOT" and ask for "!ROOT". The possible
outcomes are:

  1. we know we are root

  2. we know we are not root

  3. we could not tell, because `id` was not available

We should conservatively treat (3) as "does not have the
prerequisite", which means that a naive negation would not
work.
Helped-by: NKyle J. McKay <mackyle@gmail.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The construct is error-prone; "test" being built-in in most modern
shells, the reason to avoid "test <cond> && test <cond>" spawning
one extra process by using a single "test <cond> -a <cond>" no
longer exists.
Signed-off-by: NElia Pinto <gitter.spiros@gmail.com>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since commit 426e70d4 (remote-curl: show server content on
http errors, 2013-04-05), we relay any text/plain error
messages from the remote server to the user. However, we
never tested it.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Using write_script will set our shebang line appropriately
with $SHELL_PATH. The script that is there now is quite
simple and likely to succeed even with a non-POSIX /bin/sh,
but it does not hurt to be defensive.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Our test httpd setup will not generally run as root, because
Apache will want to setuid, and we do not set up the "User"
config directive. On some systems, like current Debian
unstable, Apache fails to start, and we skip the tests:

    $ sudo ./t5539-fetch-http-shallow.sh --debug
    1..0 # SKIP web server setup failed
    $ cat trash*t5539*/httpd/error.log
    [...]
    (22)Invalid argument: AH00024: Couldn't set permissions on
      the rewrite-map mutex; check User and Group directives
    AH00016: Configuration Failed

However, on other systems (reportedly Ubuntu 11.04), Apache
seems to start, and then bails during our tests with:

   getpwuid: couldn't determine user name from uid 4294967295,
     you probably need to modify the User directive
   Child 12037 returned a Fatal error...  Apache is exiting!

This may be related to the pre-fork/threading model in use
(note that the second one complains of the child dying).
However, it's not even worth investigating; in either case
we just want to skip the tests, and we already recommend
against running the test suite as root. Let's just
explicitly check this condition and skip the tests rather
than expecting Apache to do the right thing.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We do not run the httpd nor git-daemon tests by default, as
they are rather heavyweight and require network access
(albeit over localhost). However, it would be nice if more
pepole ran them, for two reasons:

  1. We would get more test coverage on more systems.

  2. The point of the test suite is to find regressions. It
     is very easy to change some of the underlying code and
     break the httpd code without realizing you are even
     affecting it. Running the httpd tests helps find these
     problems sooner (ideally before the patches even hit
     the list).

We still want to leave an "out", though, for people who really do
not want to run them. For that reason, the GIT_TEST_HTTPD and
GIT_TEST_GIT_DAEMON variables are now tri-state booleans
(true/false/auto), so you can say GIT_TEST_HTTPD=false to turn the
tests back off.  To support those who want a stable single way to
disable these tests across versions of Git before and after this
change, an empty string explicitly set to these variables is also
taken as "false", so the behaviour changes only for those who:

  a. did not express any preference by leaving these variables
     unset.  They did not test these features before, but now they
     do; or

  b. did express that they want to test these features by setting
     GIT_TEST_FEATURE=false (or any equivalent other ways to tell
     "false" to Git, e.g. "0"), which has been a valid but funny way
     to say that they do want to test the feature only because we
     used to interpret any non-empty string to mean "yes please
     test".  They no longer test that feature.

In addition, we are forgiving of common setup failures (e.g., you do
not have apache installed, or have an old version) when the
tri-state is "auto" (or unset), but report an error when it is
"true". This makes "auto" a sane default, as we should not cause
failures on setups where the tests cannot run. But it allows people
who use "true" to catch regressions in their system (e.g., they
uninstalled apache, but were expecting their automated test runs to
test git-httpd, and would want to be notified).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We set the default apache port for each of the httpd tests
to the 4-digit test number of the test script. We want these
to remain unique so that the tests do not conflict with each
other when run in parallel.

Instead of doing it manually in each test script, let's just
set it from the test name at run time. This is simpler, and
is one less thing to be updated when test scripts are
renamed (e.g., when being re-rolled or when conflicting
after being merged with another topic).

Incidentally, this fixes a case where t5537 and t5538 used
the same port number (5537), and could conflict with each
other when run in parallel.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The httpd server we set up to test git's http client code
knows about a single account, in which both the username and
password are "user@host" (the unusual use of the "@" here is
to verify that we handle the character correctly when URL
escaped).

This means that we may miss a certain class of errors in
which the username and password are mixed up internally by
git. We can make our tests more robust by having distinct
values for the username and password.

In addition to tweaking the server passwd file and the
client URL, we must teach the "askpass" harness to accept
multiple values. As a bonus, this makes the setup of some
tests more obvious; when we are expecting git to ask
only about the password, we can seed the username askpass
response with a bogus value.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A #! line in these files is misleading, since these scriptlets are
meant to be sourced with '.' (using whatever shell sources them)
instead of run directly using the interpreter named on the #! line.

Removing the #! line shouldn't hurt syntax highlighting since
these files have filenames ending with '.sh'.  For documentation,
add a brief description of how the files are meant to be used in
place of the shebang line.
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For efficiency and security reasons, an earlier commit in
this series taught http_get_* to re-write the base url based
on redirections we saw while making a specific request.

This commit wires that option into the info/refs request,
meaning that a redirect from

    http://example.com/foo.git/info/refs

to

    https://example.com/bar.git/info/refs

will behave as if "https://example.com/bar.git" had been
provided to git in the first place.

The tests bear some explanation. We introduce two new
hierearchies into the httpd test config:

  1. Requests to /smart-redir-limited will work only for the
     initial info/refs request, but not any subsequent
     requests. As a result, we can confirm whether the
     client is re-rooting its requests after the initial
     contact, since otherwise it will fail (it will ask for
     "repo.git/git-upload-pack", which is not redirected).

  2. Requests to smart-redir-auth will redirect, and require
     auth after the redirection. Since we are using the
     redirected base for further requests, we also update
     the credential struct, in order not to mislead the user
     (or credential helpers) about which credential is
     needed. We can therefore check the GIT_ASKPASS prompts
     to make sure we are prompting for the new location.
     Because we have neither multiple servers nor https
     support in our test setup, we can only redirect between
     paths, meaning we need to turn on
     credential.useHttpPath to see the difference.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>

Commit 05c1eb10 (push: teach --force-with-lease to smart-http
transport) fixed the compare-and-swap test in t5541. It
tried to mark the test as passing by teaching the test
helper function to expect an extra "success or failure"
parameter, but forgot to actually use the parameter in the
helper.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We have been passing enough information to enable the
compare-and-swap logic down to the transport layer, but the
transport helper was not passing it to smart-http transport.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The push() method in remote-curl.c is not told and does not pass the
necessary information to underlying send-pack, so this extension
does not yet work.  Leave a note in the test suite.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Before parsing a suspected smart-HTTP response verify the returned
Content-Type matches the standard. This protects a client from
attempting to process a payload that smells like a smart-HTTP
server response.

JGit has been doing this check on all responses since the dawn of
time. I mistakenly failed to include it in git-core when smart HTTP
was introduced. At the time I didn't know how to get the Content-Type
from libcurl. I punted, meant to circle back and fix this, and just
plain forgot about it.
Signed-off-by: NShawn Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The t5550 script sets up a nice askpass helper for
simulating user input and checking what git prompted for.
Let's make it available to other http scripts by migrating
it to lib-httpd.

We can use this immediately in t5540 to make our tests more
robust (previously, we did not check at all that hitting the
password-protected repo actually involved a password).
Unfortunately, we end up failing the test because the
current code erroneously prompts twice (once for
git-remote-http, and then again when the former spawns
git-http-push).

More importantly, though, it will let us easily add
smart-http authentication tests in t5541 and t5551; we
currently do not test smart-http authentication at all.

As part of making it generic, let's always look for and
store auxiliary askpass files at the top-level trash
directory; this makes it compatible with t5540, which runs
some tests from sub-repositories. We can abstract away the
ugliness with a short helper function.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Running the http tests with valgrind does not work for two
reasons:

  1. Apache complains about following the symbolic link from
     git-http-backend to valgrind.sh.

  2. Apache does not pass through the GIT_VALGRIND variable
     to the backend CGI.

This patch fixes both problems. Unfortunately, there is a
slight hack we need to handle passing environment variables
through Apache. If we just tell it:

  PassEnv GIT_VALGRIND

then Apache will complain when GIT_VALGRIND is not set. If
we try:

  SetEnv GIT_VALGRIND ${GIT_VALGRIND}

then when GIT_VALGRIND is not set, it will pass through the
literal "${GIT_VALGRIND}". Instead, we now unconditionally
pass through GIT_VALGRIND from lib-httpd.sh into apache,
even if it is empty.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NChristopher Tiwald <christiwald@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

These just checked that we could clone a repository when the
username and password were given in the URL; we should also
check that git will prompt when no or partial credentials
are given.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Gettextize the "To prevent you from losing history" message. A test in
lib-httpd.sh and another in t5541-http-push.sh explicitly checked for
this message. Change them to skip under GETTEXT_POISON=YesPlease.
Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add a test for HTTP authentication and proper percent-decoding of the
userinfo (username and password) part of the URL.
Signed-off-by: NGabriel Corona <gabriel.corona@enst-bretagne.fr>
Acked-by: NTay Ray Chuan <rctay89@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The '!()' notation is interpreted as a pattern-list on Ksh.  The Ksh man
page describe it as follows:

   !(pattern-list)
      Matches anything except one of the given patterns.

Ksh performs a file glob using the pattern-list and then tries to execute
the first file in the list.  If a space is added between the '!' and the
open parens, then Ksh will not interpret it as a pattern list, but in this
case, it is preferred to use test_must_fail, so lets do so.
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change tests to skip with skip_all=* + test_done instead of using say
+ test_done.

This is a follow-up to "tests: Skip tests in a way that makes sense
under TAP" (fadb5156). I missed these cases when prepearing that
patch, hopefully this is all of them.
Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

SKIP messages are now part of the TAP plan. A TAP harness now knows
why a particular test was skipped and can report that information. The
non-TAP harness built into Git's test-lib did nothing special with
these messages, and is unaffected by these changes.
Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move non-fast forward tests to lib-httpd.sh so that we don't have to
duplicate the tests in both t5540 and t5541.
Signed-off-by: NTay Ray Chuan <rctay89@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

On CentOS 5, httpd is located at /usr/sbin/httpd, and the modules are
located at /usr/lib64/httpd/modules.  To enable easy testing of httpd,
we would like those locations to be detected automatically.

uname might not be the best way to determine the default location for
httpd since different Linux distributions apparently put httpd in
different places, so we test a couple different locations for httpd,
and use the first one that we come across.  We do the same for the
modules directory.

cc: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: NTarmigan Casebolt <tarmigan+git@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Previously, die() would report the exit code of stop_httpd. Instead,
save and reset the exit code before dying.
Signed-off-by: NClemens Buchacher <drizzd@aon.at>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

test_done always exits, so this line is never executed.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

There are some redirects and some error checking that need
to be done by the caller; let's move both into the
start_httpd function so that all callers don't have to
repeat them (there is only one caller now, but another will
follow in this series).

This doesn't violate any assumptions that aren't already
being made by lib-httpd, which is happy to say "skipping"
and call test_done for a number of other cases.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJay Soffian <jaysoffian@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This patch allows the HTTP tests to run on OS X 10.5. It is not
sufficient to be able to pass in LIB_HTTPD_PATH and
LIB_HTTPD_MODULE_PATH alone, as the apache.conf also needs a couple
tweaks.

These changes are put into an <IfDefine> to keep them Darwin specific,
but this means lib-httpd.sh needs to be modified to pass -DDarwin to
apache when running on Darwin. As long as we're making this change to
lib-httpd.sh, we may as well set LIB_HTTPD_PATH and
LIB_HTTPD_MODULE_PATH to appropriate default values for the platform.

Note that we now pass HTTPD_PARA to apache at shutdown as well.
Otherwise apache will emit a harmless, but noisy warning that LogFormat
is an unknown directive.
Signed-off-by: NJay Soffian <jaysoffian@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In 'man 1p trap' there is written:

    "Implementations may permit names with the SIG prefix or ignore case
    in signal names as an extension."

So change the lowercase signals to uppercase, which is POSIX compliant
instead of being an extension.

There wasn't anybody claiming that it doesn't work, but there was a bug
with using a signal with the SIG prefix, which is an extension as well.
So let's play it safe and change it, since it doesn't hurt anyone.

While at it, also convert 8 indentation spaces to 1 tab character.
Signed-off-by: NMarkus Heidelberg <markus.heidelberg@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Many test scripts assumed that they will start in a 'trash' subdirectory
that is a single level down from the t/ directory, and referred to their
test vector files by asking for files like "../t9999/expect".  This will
break if we move the 'trash' subdirectory elsewhere.

To solve this, we earlier introduced "$TEST_DIRECTORY" so that they can
refer to t/ directory reliably.  This finally makes all the tests use
it to refer to the outside environment.

With this patch, and a one-liner not included here (because it would
contradict with what Dscho really wants to do):

| diff --git a/t/test-lib.sh b/t/test-lib.sh
| index 70ea7e0..60e69e4 100644
| --- a/t/test-lib.sh
| +++ b/t/test-lib.sh
| @@ -485,7 +485,7 @@ fi
|  . ../GIT-BUILD-OPTIONS
|
|  # Test repository
| -test="trash directory"
| +test="trash directory/another level/yet another"
|  rm -fr "$test" || {
|         trap - exit
|         echo >&5 "FATAL: Cannot prepare test area"

all the tests still pass, but we would want extra sets of eyeballs on this
type of change to really make sure.

[jc: with help from Stephan Beyer on http-push tests I do not run myself;
 credits for locating silly quoting errors go to Olivier Marin.]
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

http-push test has been broken by 4a7aaccd adding a space character
in the place where the test is being run.
Signed-off-by: NMike Hommey <mh@glandium.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>