We just introduced a test that demonstrates that our sloppy use of
regexec() on a mmap()ed area can result in incorrect results or even
hard crashes.

So what we need to fix this is a function that calls regexec() on a
length-delimited, rather than a NUL-terminated, string.

Happily, there is an extension to regexec() introduced by the NetBSD
project and present in all major regex implementation including
Linux', MacOSX' and the one Git includes in compat/regex/: by using
the (non-POSIX) REG_STARTEND flag, it is possible to tell the
regexec() function that it should only look at the offsets between
pmatch[0].rm_so and pmatch[0].rm_eo.

That is exactly what we need.

Since support for REG_STARTEND is so widespread by now, let's just
introduce a helper function that always uses it, and tell people
on a platform whose regex library does not support it to use the
one from our compat/regex/ directory.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change Makefile to include git-parse-remote.sh in LOCALIZED_SH.

TODO: remove 3rd argument of error_on_missing_default_upstream function
that is no longer required.
Signed-off-by: NVasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We don't need it, as we no longer use HMAC_CTX_cleanup() directly.
Signed-off-by: NKazuki Yamaguchi <k@rhe.jp>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In general "echo 2>&1 $msg" to redirect a possible error message
that comes from 'echo' itself into the same standard output stream
$msg is getting written to does not make any sense; it is not like
we are expecting to see any errors out of 'echo' in these statements,
and even if it were the case, there is no reason to prevent the
error messages from being sent to the standard error stream.

These are clearly meant to send the argument given to echo to the
standard error stream as error messages.  Correctly redirect by
saying "send what is written to the standard output to the standard
error", i.e. "1>&2" aka ">&2".
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Newer versions of GNU grep is reported to be pickier when we feed a
non-ASCII input and break some Porcelain scripts.  As we know we do
not feed random binary file to our own sane_grep wrapper, allow us
to always pass "-a" by setting SANE_TEXT_GREP=-a Makefile variable
to work it around.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We assume Git developers have a reasonably modern compiler and recommend
them to enable the DEVELOPER makefile knob to ensure their patches are
clear of all compiler warnings the Git core project cares about.

Enable the DEVELOPER makefile knob in the Travis-CI build.
Suggested-by: NJeff King <peff@peff.net>
Signed-off-by: NLars Schneider <larsxschneider@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since baaf2337 (connect: improve check for plink to reduce false
positives, 2015-04-26), t5601 writes out a `plink.exe` for testing that
is actually a shell script. So the assumption that the `.exe` extension
implies that the file is *not* a shell script is now wrong.

Since there was no love for the idea of allowing `.exe` files to be
shell scripts on Windows, let's go the other way round: *make*
`plink.exe` a real `.exe`.

This fixes t5601-clone.sh in Git for Windows' SDK.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add some missing phony target to Makefile.
Signed-off-by: NElia Pinto <gitter.spiros@gmail.com>
Helped-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

As another step in the move to pluggable reference backends, move the
code that is specific to the filesystem-based reference backend (i.e.,
the current system of storing references as loose and packed files) into
a separate file, refs/files-backend.c.

Aside from a tiny bit of file header boilerplate, this commit only moves
a subset of the code verbatim from refs.c to the new file, as can easily
be verified using patience diff:

    git diff --patience $commit^:refs.c $commit:refs.c
    git diff --patience $commit^:refs.c $commit:refs/files-backend.c
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

Using the previous commit's inredirection mechanism for SHA1,
support a chunked implementation of SHA1_Update() that limits the
amount of data in the chunk passed to SHA1_Update().

This is enabled by using the Makefile variable SHA1_MAX_BLOCK_SIZE
to specify chunk size.  When using Apple's CommonCrypto library this
is set to 1GiB (the implementation cannot handle more 4GiB).
Signed-off-by: NAtousa Pahlevan Duprat <apahlevan@ieee.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move the bulk of the code from builtin/mailinfo.c to mailinfo.c
so that new callers can start calling mailinfo() directly.

Note that a few calls to exit() and die() need to be cleaned up
for the API to be truly useful, which will come in later steps.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

There are situations, e.g. during cross compilation, where curl-config
program is not present in the PATH.

Make the makefile use a configurable curl-config program passed through
CURL_CONFIG variable which can be set through config.mak.

Also make this variable tunable through use of autoconf/configure. Configure
will set CURL_CONFIG variable in config.mak.autogen to whatever value has been
passed to ac_cv_prog_CURL_CONFIG.
Signed-off-by: NRemi Pommarel <repk@triplefau.lt>
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For static linking especially library order while linking is important. For
example, libcurl wants symbols from zlib when building http-push, http-fetch
and remote-curl. So for these programs libcurl has to be linked before zlib.
Signed-off-by: NRemi Pommarel <repk@triplefau.lt>
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Now that fsck has dropped its inode-sorting, there are no
longer any users of this knob, and it can go away.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

worktree.c contains functions to work with and get information from
worktrees.  This introduction moves functions related to worktrees
from branch.c into worktree.c
Signed-off-by: NMichael Rappazzo <rappazzo@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Non-POSIX shells, such as /bin/sh on SunOS, do not support $((...))
arithmetic expansion or $(...) command substitution needed by
generate-cmdlist.sh.  Make sure that we use a POSIX compliant shell
$(SHELL_PATH) when running generate-cmdlist.sh.
Signed-off-by: NAlejandro R. Sedeño <asedeno@mit.edu>
Acked-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We can do this because we have a very simple needs and run "ar"
exactly the same way everywhere ;-).

Requested-by: Jeffrey Walton
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

findstring is defined as $(findstring FIND,IN) so if multiple flags are
set these tests do the wrong thing unless $(MAKEFLAGS) is the second
argument.
Signed-off-by: NJohn Keeping <john@keeping.me.uk>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Most of the submodule operations work on a set of submodules.
Calculating and using this set is usually done via:

       module_list "$@" | {
           while read mode sha1 stage sm_path
           do
                # the actual operation
           done
       }

Currently the function `module_list` is implemented in the
git-submodule.sh as a shell script wrapping a perl script.
The rewrite is in C, such that it is faster and can later be
easily adapted when other functions are rewritten in C.

git-submodule.sh, similar to the builtin commands, will navigate
to the top-most directory of the repository and keep the
subdirectory as a variable. As the helper is called from
within the git-submodule.sh script, we are already navigated
to the root level, but the path arguments are still relative
to the subdirectory we were in when calling git-submodule.sh.
That's why there is a `--prefix` option pointing to an alternative
path which to anchor relative path arguments.
Signed-off-by: NStefan Beller <sbeller@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

527ec398 (generate-cmdlist: parse common group commands, 2015-05-21)
replaced generate-cmdlist.sh with a more functional Perl version,
generate-cmdlist.perl. The Perl version gleans named tags from a new
"common groups" section in command-list.txt and recognizes those
tags in "command list" section entries in place of the old 'common'
tag. This allows git-help to, not only recognize, but also group
common commands.

Although the tests require Perl, 527ec398 creates an unconditional
dependence upon Perl in the build system itself, which can not be
overridden with NO_PERL. Such a dependency may be undesirable; for
instance, the 'git-lite' package in the FreeBSD ports tree is
intended as a minimal Git installation (which may, for example, be
useful on servers needing only local clone and update capability),
which, historically, has not depended upon Perl[1].

Therefore, revive generate-cmdlist.sh and extend it to recognize
"common groups" and its named tags. Retire generate-cmdlist.perl.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/275905/focus=276132Signed-off-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In a superproject some commands need to interact with submodules. They
need to query values from the .gitmodules file either from the worktree
of from certain revisions. At the moment this is quite hard since a
caller would need to read the .gitmodules file from the history and then
parse the values. We want to provide an API for this so we have one
place to get values from .gitmodules from any revision (including the
worktree).

The API is realized as a cache which allows us to lazily read
.gitmodules configurations by commit into a runtime cache which can then
be used to easily lookup values from it. Currently only the values for
path or name are stored but it can be extended for any value needed.

It is expected that .gitmodules files do not change often between
commits. Thats why we lookup the .gitmodules sha1 from a commit and then
either lookup an already parsed configuration or parse and cache an
unknown one for each sha1. The cache is lazily build on demand for each
requested commit.

This cache can be used for all purposes which need knowledge about
submodule configurations. Example use cases are:

 * Recursive submodule checkout needs to lookup a submodule name from
   its path when a submodule first appears. This needs be done before
   this configuration exists in the worktree.

 * The implementation of submodule support for 'git archive' needs to
   lookup the submodule name to generate the archive when given a
   revision that is not checked out.

 * 'git fetch' when given the --recurse-submodules=on-demand option (or
   configuration) needs to lookup submodule names by path from the
   database rather than reading from the worktree. For new submodule it
   needs to lookup the name from its path to allow cloning new
   submodules into the .git folder so they can be checked out without
   any network interaction when the user does a checkout of that
   revision.
Signed-off-by: NHeiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: NStefan Beller <sbeller@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A lot of work went into defining the state diagram for lockfiles and
ensuring correct, race-resistant cleanup in all circumstances.

Most of that infrastructure can be applied directly to *any* temporary
file. So extract a new "tempfile" module from the "lockfile" module.
Reimplement lockfile on top of tempfile.

Subsequent commits will add more users of the new module.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

At the beginning of the rewrite of git-am.sh to C, in order to not break
existing test scripts that depended on a functional git-am, a
redirection to git-am.sh was introduced that would activate if the
environment variable _GIT_USE_BUILTIN_AM was not defined.

Now that all of git-am.sh's functionality has been re-implemented in
builtin/am.c, remove this redirection, and retire git-am.sh into
contrib/examples/.
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For the purpose of rewriting git-am.sh into a C builtin, implement a
skeletal builtin/am.c that redirects to $GIT_EXEC_PATH/git-am if the
environment variable _GIT_USE_BUILTIN_AM is not defined. Since in the
Makefile git-am.sh takes precedence over builtin/am.c,
$GIT_EXEC_PATH/git-am will contain the shell script git-am.sh, and thus
this allows us to fall back on the functional git-am.sh when running the
test suite for tests that depend on a working git-am implementation.

Since git-am.sh cannot handle any environment modifications by
setup_git_directory(), "am" is declared with no setup flags in git.c. On
the other hand, to re-implement git-am.sh in builtin/am.c, we need to
run all the git dir and work tree setup logic that git.c typically does
for us. As such, we work around this temporarily by copying the logic in
git.c's run_builtin(), which is roughly:

	prefix = setup_git_directory();
	trace_repo_setup(prefix);
	setup_work_tree();

This redirection should be removed when all the features of git-am.sh
have been re-implemented in builtin/am.c.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move most of the code from 'for-each-ref' to 'ref-filter' to make
it publicly available to other commands, this is to unify the code
of 'tag -l', 'branch -l' and 'for-each-ref' so that they can share
their implementations with each other.

Add 'ref-filter' to the Makefile, this completes the movement of code
from 'for-each-ref' to 'ref-filter'.
Mentored-by: NChristian Couder <christian.couder@gmail.com>
Mentored-by: NMatthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: NKarthik Nayak <karthik.188@gmail.com>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NKarsten Blees <blees@dcon.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Commit 23af91d1 (prune: strategies for linked checkouts - 2014-11-30)
adds "--worktrees" to "git prune" without realizing that "git prune" is
for object database only. This patch moves the same functionality to a
new command "git worktree".
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>

At the beginning of the rewrite of git-pull.sh to C, we introduced a
redirection to git-pull.sh if the environment variable
_GIT_USE_BUILTIN_PULL was not defined in order to not break test scripts
that relied on a functional git-pull.

Now that all of git-pull's functionality has been re-implemented in
builtin/pull.c, remove this redirection, and retire the old git-pull.sh
into contrib/examples/.
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For the purpose of rewriting git-pull.sh into a C builtin, implement a
skeletal builtin/pull.c that redirects to $GIT_EXEC_PATH/git-pull.sh if
the environment variable _GIT_USE_BUILTIN_PULL is not defined. This
allows us to fall back on the functional git-pull.sh when running the
test suite for tests that depend on a working git-pull implementation.

This redirection should be removed when all the features of git-pull.sh
have been re-implemented in builtin/pull.c.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Every time we run "make", we update perl/PM.stamp, which
contains a list of all of the perl module files (if it's
updated, we need to rebuild perl/perl.mak, since the
Makefile will not otherwise know about the new files).

This means that every time "make" is run, we see:

      GEN perl/PM.stamp

in the output, even though it is not likely to have changed.
Let's make this recipe completely silent, as we do for other
auto-generated dependency files (e.g., GIT-CFLAGS).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We force the GIT-BUILD-OPTIONS recipe to run every time
"make" is invoked. We must do this to catch new options
which may have come from the command-line or environment.

However, we actually update the file's timestamp each time
the recipe is run, whether anything changed or not. As a
result, any files which depend on it (for example, all of
the perl scripts, which need to know whether NO_PERL was
set) will be re-built every time.

Let's do our usual trick of writing to a tempfile, then
doing a "cmp || mv" to update the file only when something
changed.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The rule for "git-instaweb" depends on "gitweb". This makes
no sense, because:

  1. git-instaweb has no build-time dependency on gitweb; it
     is a run-time dependency

  2. gitweb is a directory that we want to recursively make
     in. As a result, its recipe is marked .PHONY, which
     causes "make" to rebuild git-instaweb every time it is
     run.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Parse the group block to create the array of group descriptions:

static char *common_cmd_groups[] = {
    N_("starting a working area"),
    N_("working on the current change"),
    N_("working with others"),
    N_("examining the history and state"),
    N_("growing, marking and tweaking your history"),
};

then map each element of common_cmds[] to a group via its index:

static struct cmdname_help common_cmds[] = {
    {"add", N_("Add file contents to the index"), 1},
    {"branch", N_("List, create, or delete branches"), 4},
    {"checkout", N_("Checkout a branch or paths to the ..."), 4},
    {"clone", N_("Clone a repository into a new directory"), 0},
    {"commit", N_("Record changes to the repository"), 4},
    ...
};

so that 'git help' can print those commands grouped by theme.

Only commands tagged with an attribute from the group block are emitted to
common_cmds[].

[commit message by Sébastien Guimmara <sebastien.guimmara@gmail.com>]
Signed-off-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NSébastien Guimmara <sebastien.guimmara@gmail.com>
Reviewed-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The ultimate goal is for "git help" to classify common commands by
group. Toward this end, a subsequent patch will add a new "common
groups" section to command-list.txt preceding the actual command list.
As preparation, teach existing command-list.txt parsing machinery, which
doesn't care about grouping, to skip over this upcoming "common groups"
section.
Signed-off-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NSébastien Guimmara <sebastien.guimmara@gmail.com>
Reviewed-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When built with NO_EXPAT=NoThanks, we will not have a working http-push
over webdav.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.

The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

to:

  real    0m6.768s
  user    0m6.340s
  sys     0m0.432s

for a wall-clock speedup of 21%.

Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

On BSD-compatible systems some information such as the number
of available CPUs may only be available via the sysctl function.

Add support for a HAVE_BSD_SYSCTL option complete with autoconf
support and include the sys/syctl.h header when the option is
enabled to make the sysctl function available.
Signed-off-by: NKyle J. McKay <mackyle@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

curl 7.11.0 through 7.12.2 when built from their official release
archives will present a 5 digit version number instead of the documented
6 digits which breaks the version check in the Makefile.
Correct these broken version numbers on the fly when extracting them to
ensure the comparison works correctly.

[jc: shortened the new sed scripts a bit]
Signed-off-by: NTom G. Christensen <tgc@statsbiblioteket.dk>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The gettext N_ macro is used to mark strings for translation
without actually translating them.  At runtime the string is
expected to be passed to the gettext API for translation.

If two N_ macro invocations appear next to each other with only
whitespace (or nothing at all) between them, the two separate
strings will be marked for translation, but the preprocessor
will then silently combine the strings into one and at runtime
the string passed to gettext will not match the strings that
were translated so no translation will actually occur.

Avoid this by adding parentheses around the expansion of the
N_ macro so that instead of ending up with two adjacent strings
that are then combined by the preprocessor, two adjacent strings
surrounded by parentheses result instead which causes a compile
error so the mistake can be quickly found and corrected.

However, since these string literals are typically assigned to
static variables and not all compilers support parenthesized
string literal assignments, allow this to be controlled by the
Makefile with the default only enabled when the compiler is
known to support the syntax.

For now only __GNUC__ enables this by default which covers both
gcc and clang which should result in early detection of any
adjacent N_ macros.

Although the necessary tests make the affected files a bit less
elegant, the benefit of avoiding propagation of a translation-
marking error to all the translation teams thus creating extra
work for them when the error is eventually detected and fixed
would seem to outweigh the minor inelegance the additional
configuration tests introduce.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NKyle J. McKay <mackyle@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>