Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A flaky test has been corrected.

* jk/pack-idx-corruption-safety:
  t5313: make extended-table test more deterministic

We often try to open a file for reading whose existence is
optional, and silently ignore errors from open/fopen; report such
errors if they are not due to missing files.

* nd/fopen-errors:
  mingw_fopen: report ENOENT for invalid file names
  mingw: verify that paths are not mistaken for remote nicknames
  log: fix memory leak in open_next_file()
  rerere.c: move error_errno() closer to the source system call
  print errno when reporting a system call error
  wrapper.c: make warn_on_inaccessible() static
  wrapper.c: add and use fopen_or_warn()
  wrapper.c: add and use warn_on_fopen_errors()
  config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
  config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
  clone: use xfopen() instead of fopen()
  use xfopen() in more places
  git_fopen: fix a sparse 'not declared' warning

Completion updates.

* rf/completion:
  completion: add git config credentialCache.ignoreSIGHUP
  completion: add git config credential completions
  completion: add git config advice completions
  completion: add git config am.threeWay completion
  completion: add git config core completions
  completion: add git config gc completions

Comment fix.

* jc/diff-tree-stale-comment:
  diff-tree: update stale in-code comments

Many commands learned to pay attention to submodule.recurse
configuration.

* sb/submodule-blanket-recursive:
  builtin/fetch.c: respect 'submodule.recurse' option
  builtin/push.c: respect 'submodule.recurse' option
  builtin/grep.c: respect 'submodule.recurse' option
  Introduce 'submodule.recurse' option for worktree manipulators
  submodule loading: separate code path for .gitmodules and config overlay
  reset/checkout/read-tree: unify config callback for submodule recursion
  submodule test invocation: only pass additional arguments
  submodule recursing: do not write a config variable twice

Our code often opens a path to an optional file, to work on its
contents when we can successfully open it.  We can ignore a failure
to open if such an optional file does not exist, but we do want to
report a failure in opening for other reasons (e.g. we got an I/O
error, or the file is there, but we lack the permission to open).

The exact errors we need to ignore are ENOENT (obviously) and
ENOTDIR (less obvious).  Instead of repeating comparison of errno
with these two constants, introduce a helper function to do so.

* jc/noent-notdir:
  treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked
  compat-util: is_missing_file_error()

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Docfix.

* ad/pull-remote-doc:
  docs: fix formatting and grammar

Code clean-up.

* jk/drop-free-refspecs:
  remote: drop free_refspecs() function

Leakfix.

* jk/connect-symref-info-leak-fix:
  connect.c: fix leak in parse_one_symref_info()

Completion update.

* rf/completion-config-commit:
  completion: add completions for git config commit

Test cleanup.

* ab/t3070-test-dedup:
  wildmatch test: remove redundant duplicate test

perf-test update.

* jh/memihash-opt:
  p0004: don't error out if test repo is too small
  p0004: don't abort if multi-threaded is too slow
  p0004: use test_perf
  p0004: avoid using pipes
  p0004: simplify calls of test-lazy-init-name-hash

"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.

* tb/pull-ff-rebase-autostash:
  pull: ff --rebase --autostash works in dirty repo

The timestamp of the index file is now taken after the file is
closed, to help Windows, on which a stale timestamp is reported by
fstat() on a file that is opened for writing and data was written
but not yet closed.

* jh/close-index-before-stat:
  read-cache: close index.lock in do_write_index

"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored"  did not list ignored and untracked files
without "-uall".  These have been corrected.

* sl/clean-d-ignored-fix:
  clean: teach clean -d to preserve ignored paths
  dir: expose cmp_name() and check_contains()
  dir: hide untracked contents of untracked dirs
  dir: recurse into untracked dirs for ignored files
  t7061: status --ignored should search untracked dirs
  t7300: clean -d should skip dirs with ignored files

A hotfix to a topic in 'master'.

* dk/send-email-avoid-net-smtp-ssl-when-able:
  send-email: Net::SMTP::starttls was introduced in v2.34
  send-email: Net::SMTP::SSL is obsolete, use only when necessary

A recent update to t5545-push-options.sh started skipping all the
tests in the script when a web server testing is disabled or
unavailable, not just the ones that require a web server.  Non HTTP
tests have been salvaged to always run in this script.

* jc/skip-test-in-the-middle:
  t5545: enhance test coverage when no http server is installed
  test: allow skipping the remainder

The "run-command" API implementation has been made more robust
against dead-locking in a threaded environment.

* bw/forking-and-threading:
  usage.c: drop set_error_handle()
  run-command: restrict PATH search to executable files
  run-command: expose is_executable function
  run-command: block signals between fork and execve
  run-command: add note about forking and threading
  run-command: handle dup2 and close errors in child
  run-command: eliminate calls to error handling functions in child
  run-command: don't die in child when duping /dev/null
  run-command: prepare child environment before forking
  string-list: add string_list_remove function
  run-command: use the async-signal-safe execv instead of execvp
  run-command: prepare command before forking
  t0061: run_command executes scripts without a #! line
  t5550: use write_script to generate post-update hook

Introduce the BUG() macro to improve die("BUG: ...").

* jk/bug-to-abort:
  usage: add NORETURN to BUG() function definitions
  config: complain about --local outside of a git repo
  setup_git_env: convert die("BUG") to BUG()
  usage.c: add BUG() function

"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.

* sb/checkout-recurse-submodules:
  submodule: properly recurse for read-tree and checkout
  submodule: avoid auto-discovery in new working tree manipulator code
  submodule_move_head: reuse child_process structure for futher commands

* 'master' of git://bogomips.org/git-svn:
  doc: describe git svn init --ignore-refs

Add the missing documentation for `git svn init --ignore-refs`.
Signed-off-by: NAndreas Heiduk <asheiduk@gmail.com>
Signed-off-by: NEric Wong <e@80x24.org>

* maint:
  sha1dc: update from upstream
  sha1dc: ignore indent-with-non-tab whitespace violations

* ab/sha1dc-maint:
  sha1dc: update from upstream
  sha1dc: ignore indent-with-non-tab whitespace violations

Update sha1dc from the latest version by the upstream
maintainer[1].

See commit a0103914 ("sha1dc: update from upstream", 2017-05-20) for
the latest update. That update was done sans some whitespace changes
by upstream, which is why the diff here isn't the same as the upstream
cc46554..e139984.

It also brings in a change[2] upstream made which should hopefully
address the breakage in 2.13.1 on Cygwin, see [3]. Cygwin defines both
_BIG_ENDIAN and _LITTLE_ENDIAN.

Adam Dinwoodie reports on the mailing list that that upstream commit
fixes the issue on Cygwin[4].

1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/e1399840b501a68ac6c8d7ed9a5cb1455480200e
2. https://github.com/cr-marcstevens/sha1collisiondetection/commit/a24eef58c0684078405f8c7a89f9b78271432005
3. <20170606100355.GC25777@dinwoodie.org> (https://public-inbox.org/git/20170606100355.GC25777@dinwoodie.org/)
4. <20170606124323.GD25777@dinwoodie.org> (https://public-inbox.org/git/20170606124323.GD25777@dinwoodie.org/)
Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The upstream sha1dc code indents some lines with spaces.
While this doesn't match Git's coding guidelines, it's better
to leave this imported code untouched than to try to make it
match our style. However, we can use .gitattributes to tell
"diff --check" and "git am" not to bother us about it.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Commit a1283866 (t5313: test bounds-checks of
corrupted/malicious pack/idx files, 2016-02-25) added a test
that requires our corrupted pack index to have two objects.
The entry for the first one remains untouched, but we
corrupt the entry for second one. Since the index stores the
entries in sha1-sorted order, this means that the test must
make sure that the sha1 of the object we expect to be
corrupted ("$object") sorts after the other placeholder
object.

That commit used the HEAD commit as the placeholder, but the
script never calls test_tick. That means that the commit
object (and thus its sha1) depends on the timestamp when the
test script is run. This usually works in practice, because
the sha1 of $object starts with "fff". The commit object
will sort after that only 1 in 4096 times, but when it does
the test will fail.

One obvious solution is to add the test_tick call to get a
deterministic commit sha1. But since we're relying on the
sort order for the test to function, let's make that very
explicit by just generating a second blob with a known sha1.
Reported-by: NLars Schneider <larsxschneider@gmail.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Docfix.

* ad/pull-remote-doc:
  docs: fix formatting and grammar

"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.

* tb/pull-ff-rebase-autostash:
  pull: ff --rebase --autostash works in dirty repo

Code clean-up.

* jk/drop-free-refspecs:
  remote: drop free_refspecs() function

Leakfix.

* jk/connect-symref-info-leak-fix:
  connect.c: fix leak in parse_one_symref_info()

The internal logic used in "git blame" has been libified to make it
easier to use by cgit.

* js/blame-lib: (29 commits)
  blame: move entry prepend to libgit
  blame: move scoreboard setup to libgit
  blame: move scoreboard-related methods to libgit
  blame: move fake-commit-related methods to libgit
  blame: move origin-related methods to libgit
  blame: move core structures to header
  blame: create entry prepend function
  blame: create scoreboard setup function
  blame: create scoreboard init function
  blame: rework methods that determine 'final' commit
  blame: wrap blame_sort and compare_blame_final
  blame: move progress updates to a scoreboard callback
  blame: make sanity_check use a callback in scoreboard
  blame: move no_whole_file_rename flag to scoreboard
  blame: move xdl_opts flags to scoreboard
  blame: move show_root flag to scoreboard
  blame: move reverse flag to scoreboard
  blame: move contents_from to scoreboard
  blame: move copy/move thresholds to scoreboard
  blame: move stat counters to scoreboard
  ...

The implementation of "ref" API around the "packed refs" have been
cleaned up, in preparation for further changes.

* mh/packed-ref-store-prep: (25 commits)
  cache_ref_iterator_begin(): avoid priming unneeded directories
  ref-filter: limit traversal to prefix
  create_ref_entry(): remove `check_name` option
  refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`
  read_packed_refs(): report unexpected fopen() failures
  read_packed_refs(): do more of the work of reading packed refs
  get_packed_ref_cache(): assume "packed-refs" won't change while locked
  should_pack_ref(): new function, extracted from `files_pack_refs()`
  ref_update_reject_duplicates(): add a sanity check
  ref_update_reject_duplicates(): use `size_t` rather than `int`
  ref_update_reject_duplicates(): expose function to whole refs module
  ref_transaction_prepare(): new optional step for reference updates
  ref_transaction_commit(): check for valid `transaction->state`
  files_transaction_cleanup(): new helper function
  files_ref_store: put the packed files lock directly in this struct
  files-backend: move `lock` member to `files_ref_store`
  lockfile: add a new method, is_lock_file_locked()
  ref_store: take a `msg` parameter when deleting references
  refs: use `size_t` indexes when iterating over ref transaction updates
  refs_ref_iterator_begin(): don't check prefixes redundantly
  ...

Make the "indent" heuristics the default in "diff" and diff.indentHeuristics
configuration variable an escape hatch for those who do no want it.

* mb/diff-default-to-indent-heuristics:
  add--interactive: drop diff.indentHeuristic handling
  diff: enable indent heuristic by default
  diff: have the diff-* builtins configure diff before initializing revisions
  diff: make the indent heuristic part of diff's basic configuration

The timestamp of the index file is now taken after the file is
closed, to help Windows, on which a stale timestamp is reported by
fstat() on a file that is opened for writing and data was written
but not yet closed.

* jh/close-index-before-stat:
  read-cache: close index.lock in do_write_index