The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.

The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.

To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.

For test failures, this usually means that you want to see the detailed
trace of the failing tests.

With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.

So let's do something different when using Azure DevOps: let's run all
the tests with `--quiet` first, and only if a failure is encountered,
try to trace the commands as they are executed.

Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and Azure Pipelines even know about that
outcome, but it is not available when using the JUnit XML format for
now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.csSigned-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

It is a bit ridiculous to spin up a full-blown Perl instance (especially
on Windows, where that means spinning up a full POSIX emulation layer,
AKA the MSYS2 runtime) just to tell how large a given file is.

So let's just use the test-tool to do that job instead.

This command will also be used over the next commits, to allow for
cutting out individual test cases' verbose log from the file generated
via --verbose-log.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This tests primarily for NTFS issues, but also adds one example of an
HFS+ issue.

Thanks go to Congyi Wu for coming up with the list of examples where
NTFS would possibly equate the filename with `.gitmodules`.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJeff King <peff@peff.net>

Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The parameter to basename(3) and dirname(3) traditionally had the type
"char *", but on OpenBSD it's been "const char *" for years.  That
causes (at least) Clang to throw an incompatible-pointer-types warning
for test-path-utils, where we try to pass around pointers to these
functions.

Avoid this warning (which is fatal in DEVELOPER mode) by ignoring the
promise of OpenBSD's implementations to keep input strings unmodified
and enclosing them in POSIX-compatible wrappers.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This keeps top dir a bit less crowded. And because these programs are
for testing purposes, it makes sense that they stay somewhere in t/
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The normalize_path_copy function needs an output buffer that
is at least as long as its input (it may shrink the path,
but never expand it). However, this test program feeds it
static PATH_MAX-sized buffers, which have no relation to the
input size.

In the normalize_ceiling_entry case, we do at least check
the size against PATH_MAX and die(), but that case is even
more convoluted. We normalize into a fixed-size buffer, free
the original, and then replace it with a strdup'd copy of
the result. But normalize_path_copy explicitly allows
normalizing in-place, so we can simply do that.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This strcpy will never overflow because it's copying from
baked-in test data. But we would prefer to avoid strcpy
entirely, as it makes it harder to audit for real security
bugs.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The dirname() tests file were developed and tested on only the five
platforms available to the developer at the time, namely: Linux (both 32
and 64bit), Windows XP 32-bit (MSVC), MinGW 32-bit and Cygwin 32-bit.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/basename.html
(i.e. the POSIX spec) says, in part:

	If the string pointed to by path consists entirely of the '/'
	character, basename() shall return a pointer to the string "/".
	If the string pointed to by path is exactly "//", it is
	implementation-defined whether "/" or "//" is returned.

The thinking behind testing precise, OS-dependent output values was to
document that different setups produce different values. However, as the
test failures on MacOSX illustrated eloquently: hardcoding pretty much each
and every setup's expectations is pretty fragile.

This is not limited to the "//" vs "/" case, of course, other inputs are
also allowed to produce multiple outputs by the POSIX specs.

So let's just test for all allowed values and be done with it. This still
documents that Git cannot rely on one particular output value in those
cases, so the intention of the original tests is still met.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Unfortunately, some libgen implementations yield outcomes different
from what Git expects. For example, mingw-w64-crt provides a basename()
function, that shortens `path0/` to `path`!

So let's verify that the basename() and dirname() functions we use
conform to what Git expects.
Derived-from-code-by: NRamsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In normalize_ceiling_entry(), we test that normalized paths end with
slash, *unless* the path to be normalized was already the root
directory.

However, normalize_path_copy() does not even enforce this condition.

Even worse: on Windows, the root directory gets translated into a
Windows directory by the Bash before being passed to `git.exe` (or
`test-path-utils.exe`), which means that we cannot even know whether
the path that was passed to us was the root directory to begin with.

This issue has already caused endless hours of trying to "fix" the
MSYS2 runtime, only to break other things due to MSYS2 ensuring that
the converted path maintains the same state as the input path with
respect to any final '/'.

So let's just forget about this test. It is non-essential to Git's
operation, anyway.
Acked-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NRay Donnelly <mingw.android@gmail.com>

mingw_path was introduced in abd4284b to output a mangled path as it is
passed as an argument to main(). But the name is misleading because
mangling does not come from MinGW, but from MSYS [1]. As abd4284b does not
introduce any MSYS or MinGW specific code but just prints out argv[2] as
it is passed to main(), give the function the more generic and less
confusing name "print_path".

[1] http://www.mingw.org/wiki/Posix_path_conversionSigned-off-by: NSebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>

Some test cases are skipped on Windows by marking with POSIX prereq.
This is because arguments look like absolute paths (such as /a/b)
for regular Windows programs (*.exe executables, no bash scripts)
are changed to Windows paths (like C:/msysgit/a/b).

There is no cygpath nor equivalent on msysGit, but it is easy to
write one. New subcommand "mingw_path" is added in test-path-utils,
so that we can get the expected absolute paths on Windows. E.g.

    COMMAND LINE                        Linux output  Windows output
    ==================================  ============  ===============
    test-path-utils mingw_path /        /             C:/msysgit
    test-path-utils mingw_path /a/b/    /a/b/         C:/msysgit/a/b/

With this utility, most skipped test cases in t0060 can be turned on
to be tested correctly on Windows.
Signed-off-by: NJiang Xin <worldhello.net@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Original design of relative_path() is simple, just strip the prefix
(*base) from the absolute path (*abs).

In most cases, we need a real relative path, such as: ../foo,
../../bar.  That's why there is another reimplementation
(path_relative()) in quote.c.

Borrow some codes from path_relative() in quote.c to refactor
relative_path() in path.c, so that it could return real relative
path, and user can reuse this function without reimplementing
his/her own.  The function path_relative() in quote.c will be
substituted, and I would use the new relative_path() function when
implementing the interactive git-clean later.

Different results for relative_path() before and after this refactor:

    abs path  base path  relative (original)  relative (refactor)
    ========  =========  ===================  ===================
    /a/b      /a/b       .                    ./
    /a/b/     /a/b       .                    ./
    /a        /a/b/      /a                   ../
    /         /a/b/      /                    ../../
    /a/c      /a/b/      /a/c                 ../c
    /x/y      /a/b/      /x/y                 ../../x/y

    a/b/      a/b/       .                    ./
    a/b/      a/b        .                    ./
    a         a/b        a                    ../
    x/y       a/b/       x/y                  ../../x/y
    a/c       a/b        a/c                  ../c

    (empty)   (null)     (empty)              ./
    (empty)   (empty)    (empty)              ./
    (empty)   /a/b       (empty)              ./
    (null)    (null)     (null)               ./
    (null)    (empty)    (null)               ./
    (null)    /a/b       (segfault)           ./

You may notice that return value "." has been changed to "./".
It is because:

 * Function quote_path_relative() in quote.c will show the relative
   path as "./" if abs(in) and base(prefix) are the same.

 * Function relative_path() is called only once (in setup.c), and
   it will be OK for the return value as "./" instead of ".".
Signed-off-by: NJiang Xin <worldhello.net@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add subcommand "relative_path" in test-path-utils, and add test cases
in t0060.

Johannes tested an earlier version of this patch on Windows, and
found that some relative_path tests should be skipped on
Windows. This is because the bash on Windows rewrites arguments of
regular Windows programs, such as git and the test helpers, if the
arguments look like absolute POSIX paths. As a consequence, the
actual tests performed are not what the tests scripts expect.

The tests that need *not* be skipped are those where the two paths passed
to 'test-path-utils relative_path' have the same prefix and the result is
expected to be a relative path. This is because the rewriting changes
"/a/b" to "D:/Src/MSysGit/a/b", and when both inputs are extended the same
way, this just cancels out in the relative path computation.
Signed-off-by: NJiang Xin <worldhello.net@gmail.com>
Helped-by: NJohannes Sixt <j6t@kdbg.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move the responsibility for normalizing prefixes from
longest_ancestor_length() to its callers. Use slightly different
normalizations at the two callers:

In setup_git_directory_gently_1(), use the old normalization, which
ignores paths that are not usable.  In the next commit we will change
this caller to also resolve symlinks in the paths from
GIT_CEILING_DIRECTORIES as part of the normalization.

In "test-path-utils longest_ancestor_length", use the old
normalization, but die() if any paths are unusable.  Also change t0060
to only pass normalized paths to the test program (no empty entries or
non-absolute paths, strip trailing slashes from the paths, and remove
tests that thereby become redundant).

The point of this change is to reduce the scope of the ancestor_length
tests in t0060 from testing normalization+longest_prefix to testing
only mostly longest_prefix.  This is necessary because when
setup_git_directory_gently_1() starts resolving symlinks as part of
its normalization, it will not be reasonable to do the same in the
test suite, because that would make the test results depend on the
contents of the root directory of the filesystem on which the test is
run.  HOWEVER: under Windows, bash mangles arguments that look like
absolute POSIX paths into DOS paths.  So we have to retain the level
of normalization done by normalize_path_copy() to convert the
bash-mangled DOS paths (which contain backslashes) into paths that use
forward slashes.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

Change longest_ancestor_length() to take the prefixes argument as a
string_list rather than as a colon-separated string.  This will make
it easier for the caller to alter the entries before calling
longest_ancestor_length().
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Rename the make_*_path functions so it's clearer what they do, in
particlar make clear what the differnce between make_absolute_path and
make_nonrelative_path is by renaming them real_path and absolute_path
respectively. make_relative_path has an understandable name and is
renamed to relative_path to maintain the name convention.

The function calls have been replaced 1-to-1 in their usage.
Signed-off-by: NCarlos Martín Nieto <cmn@elego.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The function strip_path_suffix() will try to strip a given suffix from
a given path.  The suffix must start at a directory boundary (i.e. "core"
is not a path suffix of "libexec/git-core", but "git-core" is).

Arbitrary runs of directory separators ("slashes") are assumed identical.

Example:

	strip_path_suffix("C:\\msysgit/\\libexec\\git-core",
		"libexec///git-core", &prefix)

will set prefix to "C:\\msysgit" and return 0.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Acked-by: NJohannes Sixt <j6t@kdbg.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This changes the test-path-utils utility to invoke normalize_path_copy()
instead of normalize_absolute_path() because the latter is about to be
removed.

The test cases in t0060 are adjusted in two regards:

- normalize_path_copy() more often leaves a trailing slash in the result.
  This has no negative side effects because the new user of this function,
  longest_ancester_length(), already accounts for this behavior.

- The function can fail.

The tests uncover a flaw in normalize_path_copy(): If there are
sufficiently many '..' path components so that the root is reached, such as
in "/d1/s1/../../d2", then the leading slash was lost. This manifested
itself that (assuming there is a repository at /tmp/foo)

  $ git add /d1/../tmp/foo/some-file

reported 'pathspec is outside repository'. This is now fixed.

Moreover, the test case descriptions of t0060 now include the test data and
expected outcome.
Signed-off-by: NJohannes Sixt <j6t@kdbg.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Previously, this test utility happily returned with exit code 0 if garbage
was thrown at it. Now it reports failure if an unknown function name was
given on the command line.
Signed-off-by: NJohannes Sixt <j6t@kdbg.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When normalizing an absolute path, we might have to add a slash _and_ a
NUL to the buffer, so the buffer was one too small.

Let's just future proof the code and alloc PATH_MAX + 1 bytes.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Make git recognize a new environment variable that prevents it from
chdir'ing up into specified directories when looking for a GIT_DIR.
Useful for avoiding slow network directories.

For example, I use git in an environment where homedirs are automounted
and "ls /home/nonexistent" takes about 9 seconds.  Setting
GIT_CEILING_DIRS="/home" allows "git help -a" (for bash completion) and
"git symbolic-ref" (for my shell prompt) to run in a reasonable time.
Signed-off-by: NDavid Reiss <dreiss@facebook.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NDavid Reiss <dreiss@facebook.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

normalize_absolute_path removes several oddities form absolute paths,
giving nice clean paths like "/dir/sub1/sub2".  Also add a test case
for this utility, based on a new test program (in the style of test-sha1).
Signed-off-by: NDavid Reiss <dreiss@facebook.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>