Code cleanup.

* rs/cocci:
  use strbuf_add_unique_abbrev() for adding short hashes, part 3
  remove unnecessary NULL check before free(3)
  coccicheck: make transformation for strbuf_addf(sb, "...") more precise
  use strbuf_add_unique_abbrev() for adding short hashes, part 2
  use strbuf_addstr() instead of strbuf_addf() with "%s", part 2
  gitignore: ignore output files of coccicheck make target
  use strbuf_addstr() for adding constant strings to a strbuf, part 2
  add coccicheck make target
  contrib/coccinelle: fix semantic patch for oid_to_hex_r()

A bit more comments in a tricky code.

* jc/diff-unique-abbrev-comments:
  diff_unique_abbrev(): document its assumption and limitation

Small doc update.

* rs/pretty-format-color-doc-fix:
  pretty: fix document link for color specification

Message fix-up.

* js/reset-usage:
  reset: fix usage

Some AsciiDoc formatter mishandles a displayed illustration with
tabs in it.  Adjust a few of them in merge-base documentation to
work around them.

* po/fix-doc-merge-base-illustration:
  doc: fix the 'revert a faulty merge' ASCII art tab spacing
  doc: fix merge-base ASCII art tab spacing

The Travis CI configuration we ship ran the tests with --verbose
option but this risks non-TAP output that happens to be "ok" to be
misinterpreted as TAP signalling a test that passed.  This resulted
in unnecessary failure.  This has been corrected by introducing a
new mode to run our tests in the test harness to send the verbose
output separately to the log file.

* jk/tap-verbose-fix:
  test-lib: bail out when "-v" used under "prove"
  travis: use --verbose-log test option
  test-lib: add --verbose-log option
  test-lib: handle TEST_OUTPUT_DIRECTORY with spaces

A hot-fix for a test added by a recent topic that went to both
'master' and 'maint' already.

* tg/add-chmod+x-fix:
  t3700: fix broken test under !SANITY

Recent git allows submodule.<name>.branch to use a special token
"." instead of the branch name; the documentation has been updated
to describe it.

* bw/submodule-branch-dot-doc:
  submodules doc: update documentation for "." used for submodule branches

Protect our code from over-eager compilers.

* jk/tighten-alloc:
  inline xalloc_flex() into FLEXPTR_ALLOC_MEM
  avoid pointer arithmetic involving NULL in FLEX_ALLOC_MEM

When fetching from a remote that has many tags that are irrelevant
to branches we are following, we used to waste way too many cycles
when checking if the object pointed at by a tag (that we are not
going to fetch!) exists in our repository too carefully.

* jk/fetch-quick-tag-following:
  fetch: use "quick" has_sha1_file for tag following

"git rebase" immediately after "git clone" failed to find the fork
point from the upstream.

* jk/merge-base-fork-point-without-reflog:
  merge-base: handle --fork-point without reflog

In a worktree connected to a repository elsewhere, created via "git
worktree", "git checkout" attempts to protect users from confusion
by refusing to check out a branch that is already checked out in
another worktree.  However, this also prevented checking out a
branch, which is designated as the primary branch of a bare
reopsitory, in a worktree that is connected to the bare
repository.  The check has been corrected to allow it.

* dk/worktree-dup-checkout-with-bare-is-ok:
  worktree: allow the main brach of a bare repository to be checked out

The "submodule.<name>.path" stored in .gitmodules is never copied
to .git/config and such a key in .git/config has no meaning, but
the documentation described it and submodule.<name>.url next to
each other as if both belong to .git/config.  This has been fixed.

* sb/submodule-config-doc-drop-path:
  documentation: improve submodule.<name>.{url, path} description

A stray symbolic link in $GIT_DIR/refs/ directory could make name
resolution loop forever, which has been corrected.

* jk/ref-symlink-loop:
  files_read_raw_ref: prevent infinite retry loops in general
  files_read_raw_ref: avoid infinite loop on broken symlinks

Documentation for "git commit" was updated to clarify that "commit
-p <paths>" adds to the current contents of the index to come up
with what to commit.

* nd/commit-p-doc:
  git-commit.txt: clarify --patch mode with pathspec

"git clone" of a local repository can be done at the filesystem
level, but the codepath did not check errors while copying and
adjusting the file that lists alternate object stores.

* jk/clone-copy-alternates-fix:
  clone: detect errors in normalize_path_copy

http.emptyauth configuration is a way to allow an empty username to
pass when attempting to authenticate using mechanisms like
Kerberos.  We took an unspecified (NULL) username and sent ":"
(i.e. no username, no password) to CURLOPT_USERPWD, but did not do
the same when the username is explicitly set to an empty string.

* dt/http-empty-auth:
  http: http.emptyauth should allow empty (not just NULL) usernames

The ./configure script generated from configure.ac was taught how
to detect support of SSL by libcurl better.

* dp/autoconf-curl-ssl:
  ./configure.ac: detect SSL in libcurl using curl-config

When we started cURL to talk to imap server when a new enough
version of cURL library is available, we forgot to explicitly add
imap(s):// before the destination.  To some folks, that didn't work
and the library tried to make HTTP(s) requests instead.

* ak/curl-imap-send-explicit-scheme:
  imap-send: Tell cURL to use imap:// or imaps://

When "git fetch" tries to find where the history of the repository
it runs in has diverged from what the other side has, it has a
mechanism to avoid digging too deep into irrelevant side branches.
This however did not work well over the "smart-http" transport due
to a design bug, which has been fixed.

* jt/fetch-pack-in-vain-count-with-stateless:
  fetch-pack: do not reset in_vain on non-novel acks

A follow-up to an already graduated topic.

* js/regexec-buf:
  configure.ac: improve description of NO_REGEX test

When "%C(auto)" appears at the very beginning of the pretty format
string, it did not need to issue the reset sequence, but it did.
This is a small optimization to already graduated topic.

* rs/c-auto-resets-attributes:
  pretty: avoid adding reset for %C(auto) if output is empty
  pretty: let %C(auto) reset all attributes

Shorten description of auto-following in "git tag" by removing a
mention of historical remotes layout which is not relevant to the
main topic.

* yk/git-tag-remove-mention-of-old-layout-in-doc:
  doc: remove reference to the traditional layout in git-tag.txt

The asciidoctor doc-tool stack does not always respect the 'tab = 8 spaces' rule
expectation, particularly for the Git-for-Windows generated html pages. This
follows on from the 'doc: fix merge-base ASCII art tab spacing' fix.

Use just spaces within the block of the ascii art.

All other *.txt ascii art containing three dashes has been checked.
Asciidoctor correctly formats the other art blocks that do contain tabs.

Signed-off-by: Philip Oakley <philipoakley@iee.org
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When there is a TAP harness consuming the output of our test
scripts, the "--verbose" breaks the output by mingling
test command output with TAP. Because the TAP::Harness
module used by "prove" is fairly lenient, this _usually_
works, but it violates the spec, and things get very
confusing if the commands happen to output a line that looks
like TAP (e.g., the word "ok" on its own line).

Let's detect this situation and complain. Just calling
error() isn't great, though; prove will tell us that the
script failed, but the message doesn't make it through to
the user. Instead, we can use the special TAP signal "Bail
out!". This not only shows the message to the user, but
instructs the harness to stop running the tests entirely.
This is exactly what we want here, as the problem is in the
command-line options, and every test script would produce
the same error.

The result looks like this (the first "Bailout called" line
is in red if prove uses color on your terminal):

 $ make GIT_TEST_OPTS='--verbose --tee'
 rm -f -r 'test-results'
 *** prove ***
 Bailout called.  Further testing stopped:  verbose mode forbidden under TAP harness; try --verbose-log
 FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log
 Makefile:39: recipe for target 'prove' failed
 make: *** [prove] Error 255
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Because we run the tests via "prove", the output from
"--verbose" may interfere with our TAP output. Using
"--verbose-log" solves this while letting us retain our
on-disk log.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:

  ./t5547-push-quarantine.sh -v | less

But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:

  $ prove t5547-push-quarantine.sh :: -v
  t5547-push-quarantine.sh .. 1/? To dest.git
   * [new branch]      HEAD -> master
  To dest.git
   ! [remote rejected] reject -> reject (pre-receive hook declined)
  error: failed to push some refs to 'dest.git'
  fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
  t5547-push-quarantine.sh .. Failed -1/4 subtests

  Test Summary Report
  -------------------
  t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
    Parse errors: Tests out of sequence.  Found (2) but expected (3)
                  Tests out of sequence.  Found (3) but expected (4)
                  Tests out of sequence.  Found (4) but expected (5)
                  Bad plan.  You planned 4 tests but ran 5.
  Files=1, Tests=5,  0 wallclock secs ( 0.01 usr +  0.01 sys =  0.02 CPU)
  Result: FAIL

One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.

Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:

 1. Add an option to write verbose output to stderr. This is
    hard to capture, though, because we want each test to
    have its own log (because they're all run in parallel
    and the jumbled output would be useless).

 2. Add an option to write verbose output to a file in
    test-results. This works, but the log is missing all of
    the non-verbose output, which gives context.

 3. Like (2), but teach say_color() to additionally output
    to the log. This mostly works, but misses any output
    that happens outside of the say() functions (which isn't
    a lot, but is a potential maintenance headache).

 4. Like (2), but make the log file the same as the "--tee"
    file. That almost works, but now we have two processes
    opening the same file. That gives us two separate
    descriptors, each with their own idea of the current
    position. They'll each start writing at offset 0, and
    overwrite each other's data.

 5. Like (4), but in each case open the file for appending.
    That atomically positions each write at the end of the
    file.

    It's possible we may still get sheared writes between
    the two processes, but this is already the case when
    writing to stdout. It's not a problem in practice
    because the test harness generally waits for snippets to
    finish before writing the TAP output.

    We can ignore buffering issues with tee, because POSIX
    mandates that it does not buffer. Likewise, POSIX
    specifies "tee -a", so it should be available
    everywhere.

This patch implements option (5), which seems to work well
in practice.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We are careful in test_done to handle a results directory
with a space in it, but the "--tee" code path does not.
Doing:

  export TEST_OUTPUT_DIRECTORY='/tmp/path with spaces'
  ./t000-init.sh --tee

results in errors. Let's consistently double-quote our path
variables so that this works.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The doc-tool stack does not always respect the 'tab = 8 spaces' rule,
particularly the git-scm doc pages https://git-scm.com/docs/git-merge-base
and the Git generated html pages.

Use just spaces within the block of the ascii art.

Noticed when reviewing Junio's suggested update to `git merge-base`
https://public-inbox.org/git/xmqqmvi2sj8f.fsf@gitster.mtv.corp.google.com/T/#uSigned-off-by: NPhilip Oakley <philipoakley@iee.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is the only place in the documentation that the traditional layout
is mentioned, and it is confusing. Remove it.

* Documentation/git-tag.txt: Here.
Signed-off-by: NYounes Khoudli <younes.khoudli@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

An "add --chmod=+x" test recently added by 610d55af ("add: modify
already added files when --chmod is given", 2016-09-14) used "xfoo3"
as a test file.  The paths xfoo[1-3] were used by earlier tests for
symbolic links but they were expected to have been removed by the
time the execution reached this new test.

The removal with "git reset --hard" however happened in a pair of
earlier tests, both of which are protected by POSIXPERM,SANITY
prerequisites.  Platforms and test environments that lacked these
would have seen xfoo3 as a leftover symbolic link that points at
somewhere else at this point of the sequence, and the chmod test
would have given a wrong result.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

4d7bc52b ("submodule update: allow '.' for branch value",
2016-08-03) adopted from Gerrit a feature to set "." as a special
value of "submodule.<name>.branch" in .gitmodules file to indicate
that the tracking branch in the submodule should be the same as the
current branch in the superproject.

Update the documentation to describe this.
Signed-off-by: NBrandon Williams <bmwill@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Allocate and copy directly in FLEXPTR_ALLOC_MEM and remove the now
unused helper function xalloc_flex().  The resulting code is shorter
and the offset arithmetic is a bit simpler.
Suggested-by: NJeff King <peff@peff.net>
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Calculating offsets involving a NULL pointer is undefined.  It works in
practice (for now?), but we should not rely on it.  Allocate first and
then simply refer to the flexible array member by its name instead of
performing pointer arithmetic up front.  The resulting code is slightly
shorter, easier to read and doesn't rely on undefined behaviour.

NB: The cast to a (non-const) void pointer is necessary to keep support
for flexible array members declared as const.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

l10n-2.10.0-rnd2.4

* tag 'l10n-2.10.0-rnd2.4' of git://github.com/git-l10n/git-po:
  l10n: de.po: translate 260 new messages
  l10n: de.po: fix translation of autostash
  l10n: ru.po: update Russian translation

* 'russian-l10n' of https://github.com/DJm00n/git-po-ru:
  l10n: ru.po: update Russian translation

When we auto-follow tags in a fetch, we look at all of the
tags advertised by the remote and fetch ones where we don't
already have the tag, but we do have the object it peels to.
This involves a lot of calls to has_sha1_file(), some of
which we can reasonably expect to fail. Since 45e8a748
(has_sha1_file: re-check pack directory before giving up,
2013-08-30), this may cause many calls to
reprepare_packed_git(), which is potentially expensive.

This has gone unnoticed for several years because it
requires a fairly unique setup to matter:

  1. You need to have a lot of packs on the client side to
     make reprepare_packed_git() expensive (the most
     expensive part is finding duplicates in an unsorted
     list, which is currently quadratic).

  2. You need a large number of tag refs on the server side
     that are candidates for auto-following (i.e., that the
     client doesn't have). Each one triggers a re-read of
     the pack directory.

  3. Under normal circumstances, the client would
     auto-follow those tags and after one large fetch, (2)
     would no longer be true. But if those tags point to
     history which is disconnected from what the client
     otherwise fetches, then it will never auto-follow, and
     those candidates will impact it on every fetch.

So when all three are true, each fetch pays an extra
O(nr_tags * nr_packs^2) cost, mostly in string comparisons
on the pack names. This was exacerbated by 47bf4b0f
(prepare_packed_git_one: refactor duplicate-pack check,
2014-06-30) which uses a slightly more expensive string
check, under the assumption that the duplicate check doesn't
happen very often (and it shouldn't; the real problem here
is how often we are calling reprepare_packed_git()).

This patch teaches fetch to use HAS_SHA1_QUICK to sacrifice
accuracy for speed, in cases where we might be racy with a
simultaneous repack. This is similar to the fix in 0eeb077b
(index-pack: avoid excessive re-reading of pack directory,
2015-06-09). As with that case, it's OK for has_sha1_file()
occasionally say "no I don't have it" when we do, because
the worst case is not a corruption, but simply that we may
fail to auto-follow a tag that points to it.

Here are results from the included perf script, which sets
up a situation similar to the one described above:

Test            HEAD^               HEAD
----------------------------------------------------------
5550.4: fetch   11.21(10.42+0.78)   0.08(0.04+0.02) -99.3%
Reported-by: NVegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In bare repositories, get_worktrees() still returns the main repository,
so git worktree list can show it. ignore it in find_shared_symref so we
can still check out the main branch.
Signed-off-by: NDennis Kaarsemaker <dennis@kaarsemaker.net>
Acked-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The --fork-point option looks in the reflog to try to find
where a derived branch forked from a base branch. However,
if the reflog for the base branch is totally empty (as it
commonly is right after cloning, which does not write a
reflog entry), then our for_each_reflog call will not find
any entries, and we will come up with no merge base, even
though there may be one with the current tip of the base.

We can fix this by just adding the current tip to
our list of collected entries.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Translate 260 new message came from git.pot updates in 9fa976ff (l10n:
git.pot: v2.10.0 round 1 (248 new, 56 removed)) and 5bd166d8 (l10n:
git.pot: v2.10.0 round 2 (12 new, 44 removed)).
Signed-off-by: NRalf Thielow <ralf.thielow@gmail.com>
Acked-by: NMatthias Rüster <matthias.ruester@gmail.com>