As with the recent speedup to strbuf_addch, we can avoid
calling strbuf_grow() in a tight loop of single-character
adds by instead checking strbuf_avail.

Note that we would instead call strbuf_addch directly here,
but it does more work than necessary: it will NUL-terminate
the result for each character read. Instead, in this loop we
read the characters one by one and then add the terminator
manually at the end.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m10.948s
  user    0m10.548s
  sys     0m0.412s

to:

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

for a wall-clock speedup of 21%.
Helped-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We mark strbuf_addch as inline, because we expect it may be
called from a tight loop. However, the first thing it does
is call the non-inline strbuf_grow(), which can handle
arbitrary-sized growth. Since we know that we only need a
single character, we can use the inline strbuf_avail() to
quickly check whether we need to grow at all.

Our check is redundant when we do call strbuf_grow(), but
that's OK. The common case is that we avoid calling it at
all, and we have made that case faster.

On a silly pathological case:

  perl -le '
    print "[core]";
    print "key$_ = value$_" for (1..1000000)
  ' >input
  git config -f input core.key1

this dropped the time to run git-config from:

  real    0m0.159s
  user    0m0.152s
  sys     0m0.004s

to:

  real    0m0.140s
  user    0m0.136s
  sys     0m0.004s

for a savings of 12%.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We read config files character-by-character from a stdio
handle using fgetc(). This incurs significant locking
overhead, even though we know that only one thread can
possibly access the handle. We can speed this up by taking
the lock ourselves, and then using getc_unlocked to read
each character.

On a silly pathological case:

  perl -le '
    print "[core]";
    print "key$_ = value$_" for (1..1000000)
  ' >input
  git config -f input core.key1

this dropped the time to run git-config from:

  real    0m0.263s
  user    0m0.260s
  sys     0m0.000s

to:

  real    0m0.159s
  user    0m0.152s
  sys     0m0.004s

for a savings of 39%.  Most config files are not this big,
but the savings should be proportional to the size of the
file (i.e., we always save 39%, just of a much smaller
number).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

strbuf_getwholeline calls getc in a tight loop. On modern
libc implementations, the stdio code locks the handle for
every operation, which means we are paying a significant
overhead.  We can get around this by locking the handle for
the whole loop and using the unlocked variant.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

to:

  real    0m10.953s
  user    0m10.384s
  sys     0m0.580s

for a wall-clock speedup of 42%. All times are best-of-3,
and done on a glibc 2.19 system.

Note that we call into strbuf_grow while holding the lock.
It's possible for that function to call other stdio
functions (e.g., printing to stderr when dying due to malloc
error); however, the POSIX.1-2001 definition of flockfile
makes it clear that the locks are per-handle, so we are fine
unless somebody else tries to read from our same handle.
This doesn't ever happen in the current code, and is
unlikely to be added in the future (we would have to do
something exotic like add a die_routine that tried to read
from stdin).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

POSIX.1-2001 specifies some functions for optimizing the
locking out of tight getc() loops. Not all systems are
POSIX, though, and even not all POSIX systems are required
to implement these functions. We can check for the
feature-test macro to see if they are available, and if not,
provide a noop implementation.

There's no Makefile knob here, because we should just detect
this automatically. If there are very bizarre systems, we
may need to add one, but it's not clear yet in which
direction:

  1. If a system defines _POSIX_THREAD_SAFE_FUNCTIONS but
     these functions are missing or broken, we would want a
     knob to manually turn them off.

  2. If a system has these functions but does not define
     _POSIX_THREAD_SAFE_FUNCTIONS, we would want a knob to
     manually turn them on.

We can add such a knob when we find a real-world system that
matches this.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

strbuf_getwholeline calls fgetc in a tight loop. Using the
getc form, which can be implemented as a macro, should be
faster (and we do not care about it evaluating our argument
twice, as we just have a plain variable).

On my glibc system, running "git rev-parse
refs/heads/does-not-exist" on a file with an extremely large
(1.6GB) packed-refs file went from (best of 3 runs):

  real    0m19.383s
  user    0m18.876s
  sys     0m0.528s

to:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

for a wall-clock speedup of 2.5%.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

People often forget to chain the commands in their test together
with &&, leaving a failure from an earlier command in the test go
unnoticed.  The new GIT_TEST_CHAIN_LINT mechanism allows you to
catch such a mistake more easily.

* jk/test-chain-lint: (36 commits)
  t9001: drop save_confirm helper
  t0020: use test_* helpers instead of hand-rolled messages
  t: simplify loop exit-code status variables
  t: fix some trivial cases of ignored exit codes in loops
  t7701: fix ignored exit code inside loop
  t3305: fix ignored exit code inside loop
  t0020: fix ignored exit code inside loops
  perf-lib: fix ignored exit code inside loop
  t6039: fix broken && chain
  t9158, t9161: fix broken &&-chain in git-svn tests
  t9104: fix test for following larger parents
  t4104: drop hand-rolled error reporting
  t0005: fix broken &&-chains
  t7004: fix embedded single-quotes
  t0050: appease --chain-lint
  t9001: use test_when_finished
  t4117: use modern test_* helpers
  t6034: use modern test_* helpers
  t1301: use modern test_* helpers
  t0020: use modern test_* helpers
  ...

Code clean-up.

* sg/completion-gitcomp-nl-for-refs:
  completion: use __gitcomp_nl() for completing refs

Code clean-up.

* jc/report-path-error-to-dir:
  report_path_error(): move to dir.c

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Doc clean-up.

* nd/doc-git-index-version:
  git.txt: list index versions in plain English

The run-command interface was easy to abuse and make a pipe for us
to read from the process, wait for the process to finish and then
attempt to read its output, which is a pattern that lead to a
deadlock.  Fix such uses by introducing a helper to do this
correctly (i.e. we need to read first and then wait the process to
finish) and also add code to prevent such abuse in the run-command
helper.

* jk/run-command-capture:
  run-command: forbid using run_command with piped output
  trailer: use capture_command
  submodule: use capture_command
  wt-status: use capture_command
  run-command: introduce capture_command helper
  wt_status: fix signedness mismatch in strbuf_read call
  wt-status: don't flush before running "submodule status"

A test fix.

* tg/test-index-v4:
  t1700: make test pass with index-v4

"git prune" used to largely ignore broken refs when deciding which
objects are still being used, which could spread an existing small
damage and make it a larger one.

* jk/prune-with-corrupt-refs:
  refs.c: drop curate_packed_refs
  repack: turn on "ref paranoia" when doing a destructive repack
  prune: turn on ref_paranoia flag
  refs: introduce a "ref paranoia" flag
  t5312: test object deletion code paths in a corrupted repository

The split-index mode introduced at v2.3.0-rc0~41 was broken in the
codepath to protect us against a broken reimplementation of Git
that writes an invalid index with duplicated index entries, etc.

* tg/fix-check-order-with-split-index:
  read-cache: fix reading of split index

"git fetch" that fetches a commit using the allow-tip-sha1-in-want
extension could have failed to fetch all the requested refs.

* jk/fetch-pack:
  fetch-pack: remove dead assignment to ref->new_sha1
  fetch_refs_via_pack: free extra copy of refs
  filter_ref: make a copy of extra "sought" entries
  filter_ref: avoid overwriting ref->old_sha1 with garbage

An failure early in the "git clone" that started creating the
working tree and repository could have resulted in some directories
and files left without getting cleaned up.

* jk/cleanup-failed-clone:
  clone: drop period from end of die_errno message
  clone: initialize atexit cleanup handler earlier

Recommend format-patch and send-email for those who want to submit
patches to this project.

* jc/submitting-patches-mention-send-email:
  SubmittingPatches: encourage users to use format-patch and send-email

"git log --graph --no-walk A B..." is a otcnflicting request that
asks nonsense; no-walk tells us show discrete points in the
history, while graph asks to draw connections between these
discrete points. Forbid the combination.

* dj/log-graph-with-no-walk:
  revision: forbid combining --graph and --no-walk

"git rev-list --bisect --first-parent" does not work (yet) and can
even cause SEGV; forbid it.  "git log --bisect --first-parent"
would not be useful until "git bisect --first-parent" materializes,
so it is also forbidden for now.

* kd/rev-list-bisect-first-parent:
  rev-list: refuse --first-parent combined with --bisect

Even though "git grep --quiet" is run merely to ask for the exit
status, we spawned the pager regardless.  Stop doing that.

* ws/grep-quiet-no-pager:
  grep: fix "--quiet" overwriting current output

Code simplification.

* jk/simplify-csum-file-sha1fd-check:
  sha1fd_check: die when we cannot open the file

The prompt script (in contrib/) did not show the untracked sign
when working in a subdirectory without any untracked files.

* ct/prompt-untracked-fix:
  git prompt: use toplevel to find untracked files

The idea of this helper is that we want to save the current
value of a config variable and then restore it again after
the test completes. However, there's no point in actually
saving the value; it should always be restored to the string
"never" (which you can confirm by instrumenting
save_confirm to print the value it finds).

Let's just replace it with a single test_when_finished call.
Suggested-by: NSZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

These tests are not wrong, but it is much shorter and more
idiomatic to say "verbose" or "test_must_fail" rather than
printing our own messages on failure. Likewise, there is no
need to say "happy" at the end of a test; the test suite
takes care of that.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since shell loops may drop the exit code of failed commands
inside the loop, some tests try to keep track of the status
by setting a variable. This can end up cumbersome and hard
to read; it is much simpler to just exit directly from the
loop using "return 1" (since each case is either in a helper
function or inside a test snippet).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

These are all cases where we do a setup step of the form:

  for i in $foo; do
	  set_up $i || break
  done &&
  more_setup

would not notice a failure in set_up (because break always
returns a 0 exit code). These are just setup steps that we
do not expect to fail, but it does not hurt to be defensive.

Most can be fixed by converting the "break" to a "return 1"
(since we eval our tests inside a function for just this
purpose). A few of the loops are inside subshells, so we can
use just "exit 1" to break out of the subshell. And a few
can actually be made shorter by just unrolling the loop.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When checking a list of file mtimes, we use a loop and break
out early from the loop if any entry does not match.
However, the exit code of a loop exited via break is always
0, meaning that the test will fail to notice we had a
mismatch. Since the loop is inside a function, we can fix
this by doing an early "return 1".
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When we test deleting notes, we run "git notes remove" in a
loop. However, the exit value of the loop will only reflect
the final note we process. We should break out of the loop
with a failing exit code as soon as we see a problem.

Note that we can call "exit 1" here without explicitly
creating a subshell, because the while loop on the
right-hand side of a pipe executes in its own implicit
subshell.

Note also that the "break" above does not suffer the same
problem; it is meant to exit the loop early at a certain
number of iterations. We can bump it into the conditional of
the loop to make this more obvious.
Signed-off-by: NJeff King <peff@peff.net>
Acked-by: NJohan Herland <johan@herland.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A loop like:

  for f in one two; do
	  something $f ||
	  break
  done

will correctly break out of the loop when we see a failure
of one item, but the resulting exit code will always be
zero. We can fix that by putting the loop into a function or
subshell, but in this case it is simpler still to just
unroll the loop. We do add a helper function, which
hopefully makes the end result even more readable (in
addition to being shorter).
Reported-by: NSZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When copying the test repository, we try to detect whether
the copy succeeded. However, most of the heavy lifting is
done inside a for loop, where our "break" will lose the exit
code of the failing "cp". We can take advantage of the fact
that we are in a subshell, and just "exit 1" to break out
with a code.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

* 'master' of git://ozlabs.org/~paulus/gitk:
  gitk: Update .po files
  gitk: l10n: Add Catalan translation
  gitk: Fix typo in Russian translation
  gitk: Remove tcl-format flag from a message that shouldn't have it
  gitk: Pass --invert-grep option down to "git log"
  gitk: Synchronize config file writes
  gitk: Report errors in saving config file
  gitk: Only write changed configuration variables
  gitk: Enable mouse horizontal scrolling in diff pane
  gitk: Default wrcomcmd to use --pretty=email

The expected call sequence is for the caller to use match_pathspec()
repeatedly on a set of pathspecs, accumulating the "hits" in a
separate array, and then call this function to diagnose a pathspec
that never matched anything, as that can indicate a typo from the
command line, e.g. "git commit Maekfile".

Many builtin commands use this function from builtin/ls-files.c,
which is not a very healthy arrangement.  ls-files might have been
the first command to feel the need for such a helper, but the need
is shared by everybody who uses the "match and then report" pattern.

Move it to dir.c where match_pathspec() is defined.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

At the first look, a user may think the default version is "23". Even
with UNIX background, there's no reference anywhere close that may
indicate this is glob or regex.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Hopefully with another batch or two, we would be ready for -rc0
to close this cycle.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The code that reads from the ctags file in the completion script
(in contrib/) did not spell ${param/pattern/string} substitution
correctly, which happened to work with bash but not with zsh.

* js/completion-ctags-pattern-substitution-fix:
  contrib/completion: escape the forward slash in __git_match_ctag

Restructure "git push" codepath to make it easier to add new
configuration bits and then add push.followTags configuration that
turns --follow-tags option on by default.

* jk/push-config:
  push: allow --follow-tags to be set by config push.followTags
  cmd_push: pass "flags" pointer to config callback
  cmd_push: set "atomic" bit directly
  git_push_config: drop cargo-culted wt_status pointer

Documentation updates.

* nd/config-doc-camelCase:
  *config.txt: stick to camelCase naming convention