git-am applies a series of patches. If the process terminates
abnormally, we want to be able to resume applying the series of patches.
This requires the session state to be saved in a persistent location.

Implement the mechanism of a "patch queue", represented by 2 integers --
the index of the current patch we are applying and the index of the last
patch, as well as its lifecycle through the following functions:

* am_setup(), which will set up the state directory
  $GIT_DIR/rebase-apply. As such, even if the process exits abnormally,
  the last-known state will still persist.

* am_load(), which is called if there is an am session in
  progress, to load the last known state from the state directory so we
  can resume applying patches.

* am_run(), which will do the actual patch application. After applying a
  patch, it calls am_next() to increment the current patch index. The
  logic for applying and committing a patch is not implemented yet.

* am_destroy(), which is finally called when we successfully applied all
  the patches in the queue, to clean up by removing the state directory
  and its contents.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Helped-by: NStefan Beller <sbeller@google.com>
Helped-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For the purpose of rewriting git-am.sh into a C builtin, implement a
skeletal builtin/am.c that redirects to $GIT_EXEC_PATH/git-am if the
environment variable _GIT_USE_BUILTIN_AM is not defined. Since in the
Makefile git-am.sh takes precedence over builtin/am.c,
$GIT_EXEC_PATH/git-am will contain the shell script git-am.sh, and thus
this allows us to fall back on the functional git-am.sh when running the
test suite for tests that depend on a working git-am implementation.

Since git-am.sh cannot handle any environment modifications by
setup_git_directory(), "am" is declared with no setup flags in git.c. On
the other hand, to re-implement git-am.sh in builtin/am.c, we need to
run all the git dir and work tree setup logic that git.c typically does
for us. As such, we work around this temporarily by copying the logic in
git.c's run_builtin(), which is roughly:

	prefix = setup_git_directory();
	trace_repo_setup(prefix);
	setup_work_tree();

This redirection should be removed when all the features of git-am.sh
have been re-implemented in builtin/am.c.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A common usage pattern of fopen() is to check if it succeeded, and die()
if it failed:

	FILE *fp = fopen(path, "w");
	if (!fp)
		die_errno(_("could not open '%s' for writing"), path);

Implement a wrapper function xfopen() for the above, so that we can save
a few lines of code and make the die() messages consistent.
Helped-by: NJeff King <peff@peff.net>
Helped-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A common usage pattern of open() is to check if it was successful, and
die() if it was not:

	int fd = open(path, O_WRONLY | O_CREAT, 0777);
	if (fd < 0)
		die_errno(_("Could not open '%s' for writing."), path);

Implement a wrapper function xopen() that does the above so that we can
save a few lines of code, and make the die() messages consistent.
Helped-by: NTorsten Bögershausen <tboegi@web.de>
Helped-by: NJeff King <peff@peff.net>
Helped-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NPaul Tan <pyokagan@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Teach "git log" and friends a new "--date=format:..." option to
format timestamps using system's strftime(3).

* jk/date-mode-format:
  strbuf: make strbuf_addftime more robust
  introduce "format" date-mode
  convert "enum date_mode" into a struct
  show-branch: use DATE_RELATIVE instead of magic number

* pt/am-tests:
  t3901: test git-am encoding conversion
  t3418: non-interactive rebase --continue with rerere enabled
  t4150: tests for am --[no-]scissors
  t4150: am with post-applypatch hook
  t4150: am with pre-applypatch hook
  t4150: am with applypatch-msg hook
  t4150: am --resolved fails if index has unmerged entries
  t4150: am --resolved fails if index has no changes
  t4150: am refuses patches when paused
  t4151: am --abort will keep dirty index intact
  t4150: am fails if index is dirty
  t4150: am.messageid really adds the message id

Optimize computation of untracked status indicator by bash prompt
script (in contrib/).

* sg/bash-prompt-untracked-optim:
  bash prompt: faster untracked status indicator with untracked directories
  bash prompt: test untracked files status indicator with untracked dirs

An experimental "untracked cache" feature used uname(2) in a
slightly unportable way.

* cb/uname-in-untracked:
  untracked: fix detection of uname(2) failure

A "rebase" replays changes of the local branch on top of something
else, as such they are placed in stage #3 and referred to as
"theirs", while the changes in the new base, typically a foreign
work, are placed in stage #2 and referred to as "ours".  Clarify
the "checkout --ours/--theirs".

* se/doc-checkout-ours-theirs:
  checkout: document subtlety around --ours/--theirs

The "rev-parse --parseopt" mode parsed the option specification
and the argument hint in a strange way to allow '=' and other
special characters in the option name while forbidding them from
the argument hint.  This made it impossible to define an option
like "--pair <key>=<value>" with "pair=key=value" specification,
which instead would have defined a "--pair=key <value>" option.

* ib/scripted-parse-opt-better-hint-string:
  rev-parse --parseopt: allow [*=?!] in argument hints

Often a fast-import stream builds a new commit on top of the
previous commit it built, and it often unconditionally emits a
"from" command to specify the first parent, which can be omitted in
such a case.  This caused fast-import to forget the tree of the
previous commit and then re-read it from scratch, which was
inefficient.  Optimize for this common case.

* mh/fast-import-optimize-current-from:
  fast-import: do less work when given "from" matches current branch head

* kn/tag-doc-fix:
  Documentation/tag: remove double occurance of "<pattern>"

"git fast-import" learned to respond to the get-mark command via
its cat-blob-fd interface.

* mh/fast-import-get-mark:
  fast-import: add a get-mark command

Add "drop commit-object-name subject" command as another way to
skip replaying of a commit in "rebase -i", and then punish those
who do not use it (and instead just remove the lines) by throwing
a warning.

* gr/rebase-i-drop-warn:
  git rebase -i: add static check for commands and SHA-1
  git rebase -i: warn about removed commits
  git-rebase -i: add command "drop" to remove a commit

Memory use reduction when commit-slab facility is used to annotate
sparsely (which is not recommended in the first place).

* jc/commit-slab:
  commit-slab: introduce slabname##_peek() function

Add a new configuration variable to enable "--follow" automatically
when "git log" is run with one pathspec argument.

* dt/log-follow-config:
  log: add "log.follow" configuration variable

Teach "git status" to show a more detailed information regarding
the "rebase -i" session in progress.

* gp/status-rebase-i-info:
  status: add new tests for status during rebase -i
  status: give more information during rebase -i
  status: differentiate interactive from non-interactive rebases
  status: factor two rebase-related messages together

"cat-file" learned "--batch-all-objects" option to enumerate all
available objects in the repository more quickly than "rev-list
--all --objects" (the output includes unreachable objects, though).

* jk/cat-file-batch-all:
  cat-file: sort and de-dup output of --batch-all-objects
  cat-file: add --batch-all-objects option
  cat-file: split batch_one_object into two stages
  cat-file: stop returning value from batch_one_object
  cat-file: add --buffer option
  cat-file: move batch_options definition to top of file
  cat-file: minor style fix in options list

Allow ignoring fsck errors on specific set of known-to-be-bad
objects, and also tweaking warning level of various kinds of non
critical breakages reported.

* js/fsck-opt:
  fsck: support ignoring objects in `git fsck` via fsck.skiplist
  fsck: git receive-pack: support excluding objects from fsck'ing
  fsck: introduce `git fsck --connectivity-only`
  fsck: support demoting errors to warnings
  fsck: document the new receive.fsck.<msg-id> options
  fsck: allow upgrading fsck warnings to errors
  fsck: optionally ignore specific fsck issues completely
  fsck: disallow demoting grave fsck errors to warnings
  fsck: add a simple test for receive.fsck.<msg-id>
  fsck: make fsck_tag() warn-friendly
  fsck: handle multiple authors in commits specially
  fsck: make fsck_commit() warn-friendly
  fsck: make fsck_ident() warn-friendly
  fsck: report the ID of the error/warning
  fsck (receive-pack): allow demoting errors to warnings
  fsck: offer a function to demote fsck errors to warnings
  fsck: provide a function to parse fsck message IDs
  fsck: introduce identifiers for fsck messages
  fsck: introduce fsck options

Clean up refs API and make "git clone" less intimate with the
implementation detail.

* mh/init-delete-refs-api:
  delete_ref(): use the usual convention for old_sha1
  cmd_update_ref(): make logic more straightforward
  update_ref(): don't read old reference value before delete
  check_branch_commit(): make first parameter const
  refs.h: add some parameter names to function declarations
  refs: move the remaining ref module declarations to refs.h
  initial_ref_transaction_commit(): check for ref D/F conflicts
  initial_ref_transaction_commit(): check for duplicate refs
  refs: remove some functions from the module's public interface
  initial_ref_transaction_commit(): function for initial ref creation
  repack_without_refs(): make function private
  prune_refs(): use delete_refs()
  prune_remote(): use delete_refs()
  delete_refs(): bail early if the packed-refs file cannot be rewritten
  delete_refs(): make error message more generic
  delete_refs(): new function for the refs API
  delete_ref(): handle special case more explicitly
  remove_branches(): remove temporary
  delete_ref(): move declaration to refs.h

Reimplement 'git pull' in C.

* pt/pull-builtin:
  pull: remove redirection to git-pull.sh
  pull --rebase: error on no merge candidate cases
  pull --rebase: exit early when the working directory is dirty
  pull: configure --rebase via branch.<name>.rebase or pull.rebase
  pull: teach git pull about --rebase
  pull: set reflog message
  pull: implement pulling into an unborn branch
  pull: fast-forward working tree if head is updated
  pull: check if in unresolved merge state
  pull: support pull.ff config
  pull: error on no merge candidates
  pull: pass git-fetch's options to git-fetch
  pull: pass git-merge's options to git-merge
  pull: pass verbosity, --progress flags to fetch and merge
  pull: implement fetch + merge
  pull: implement skeletal builtin pull
  argv-array: implement argv_array_pushv()
  parse-options-cb: implement parse_opt_passthru_argv()
  parse-options-cb: implement parse_opt_passthru()

Enhance packet tracing machinery to allow capturing an incoming
pack data to a file for debugging.

* jk/pkt-log-pack:
  pkt-line: support tracing verbatim pack contents
  pkt-line: tighten sideband PACK check when tracing
  pkt-line: simplify starts_with checks in packet tracing

"git rebase -i"'s list of todo is made configurable.

* mr/rebase-i-customize-insn-sheet:
  git-rebase--interactive.sh: add config option for custom instruction format

"git send-email" now performs alias-expansion on names that are
given via --cccmd, etc.

This round comes with a lot more enhanced e-mail address parser,
which makes it a bit scary, but as long as it works as designed, it
makes it wonderful ;-).

* rl/send-email-aliases:
  send-email: suppress meaningless whitespaces in from field
  send-email: allow multiple emails using --cc, --to and --bcc
  send-email: consider quote as delimiter instead of character
  send-email: reduce dependencies impact on parse_address_line
  send-email: minor code refactoring
  send-email: allow use of aliases in the From field of --compose mode
  send-email: refactor address list process
  t9001-send-email: refactor header variable fields replacement
  send-email: allow aliases in patch header and command script outputs
  t9001-send-email: move script creation in a setup test

* kb/i18n-doc:
  Documentation/i18n.txt: clarify character encoding support

Running an aliased command from a subdirectory when the .git thing
in the working tree is a gitfile pointing elsewhere did not work.

* nd/export-worktree:
  setup: set env $GIT_WORK_TREE when work tree is set, like $GIT_DIR

Replace "is this subdirectory a separate repository that should not
be touched?" check "git clean" does by checking if it has .git/HEAD
using the submodule-related code with a more optimized check.

* ee/clean-remove-dirs:
  read_gitfile_gently: fix use-after-free
  clean: improve performance when removing lots of directories
  p7300: add performance tests for clean
  t7300: add tests to document behavior of clean and nested git
  setup: sanity check file size in read_gitfile_gently
  setup: add gentle version of read_gitfile

Move machinery to parse human-readable scaled numbers like 1k, 4M,
and 2G as an option parameter's value from pack-objects to
parse-options API, to make it available to other codepaths.

* cb/parse-magnitude:
  parse-options: move unsigned long option parsing out of pack-objects.c
  test-parse-options: update to handle negative ints

"git verify-tag" and "git verify-commit" have been taught to share
more code, and then learned to optionally show the verification
message from the underlying GPG implementation.

* bc/gpg-verify-raw:
  verify-tag: add option to print raw gpg status information
  verify-commit: add option to print raw gpg status information
  gpg: centralize printing signature buffers
  gpg: centralize signature check
  verify-commit: add test for exit status on untrusted signature
  verify-tag: share code with verify-commit
  verify-tag: add tests

Various enhancements around "git am" reading patches generated by
foreign SCM.

* pt/am-foreign:
  am: teach mercurial patch parser how to read from stdin
  am: use gmtime() to parse mercurial patch date
  t4150: test applying StGit series
  am: teach StGit patch parser how to read from stdin
  t4150: test applying StGit patch

GSoC project to rebuild ref listing by branch and tag based on the
for-each-ref machinery.  This is its first part.

* kn/for-each-ref:
  ref-filter: make 'ref_array_item' use a FLEX_ARRAY for refname
  for-each-ref: introduce filter_refs()
  ref-filter: move code from 'for-each-ref'
  ref-filter: add 'ref-filter.h'
  for-each-ref: rename variables called sort to sorting
  for-each-ref: rename some functions and make them public
  for-each-ref: introduce 'ref_array_clear()'
  for-each-ref: introduce new structures for better organisation
  for-each-ref: rename 'refinfo' to 'ref_array_item'
  for-each-ref: clean up code
  for-each-ref: extract helper functions out of grab_single_ref()

Add an environment variable to tell Git to look into refs hierarchy
other than refs/replace/ for the object replacement data.

* mh/replace-refs:
  Allow to control where the replace refs are looked for

This would remove the need of using a pointer to store refname.
Mentored-by: NChristian Couder <christian.couder@gmail.com>
Mentored-by: NMatthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: NKarthik Nayak <karthik.188@gmail.com>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Introduce filter_refs() which will act as an API for filtering
a set of refs. Based on the type of refs the user has requested,
we iterate through those refs and apply filters as per the
given ref_filter structure and finally store the filtered refs
in the ref_array structure.

Currently this will wrap around ref_filter_handler(). Hence,
ref_filter_handler is made file scope static.

As users of this API will no longer send a ref_filter_cbdata
structure directly, we make the elements of ref_filter_cbdata
pointers. We can now use the information given by the users
to obtain our own ref_filter_cbdata structure. Changes are made to
support the change in ref_filter_cbdata structure.

Make 'for-each-ref' use this API.
Helped-by: NJunio C Hamano <gitster@pobox.com>
Mentored-by: NChristian Couder <christian.couder@gmail.com>
Mentored-by: NMatthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: NKarthik Nayak <karthik.188@gmail.com>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move most of the code from 'for-each-ref' to 'ref-filter' to make
it publicly available to other commands, this is to unify the code
of 'tag -l', 'branch -l' and 'for-each-ref' so that they can share
their implementations with each other.

Add 'ref-filter' to the Makefile, this completes the movement of code
from 'for-each-ref' to 'ref-filter'.
Mentored-by: NChristian Couder <christian.couder@gmail.com>
Mentored-by: NMatthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: NKarthik Nayak <karthik.188@gmail.com>
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Doc update.

* jk/pretty-encoding-doc:
  docs: clarify that --encoding can produce invalid sequences

Doc update.

* tb/checkout-doc:
  git-checkout.txt: document "git checkout <pathspec>" better