These calls to snprintf should always succeed, because their
input is small and fixed. Let's use xsnprintf to make sure
this is the case (and to make auditing for actual truncation
easier).

These could be candidates for turning into heap buffers, but
they fall into a few broad categories that make it not worth
doing:

  - formatting single numbers is simple enough that we can
    see the result should fit

  - the size of a sha1 is likewise well-known, and I didn't
    want to cause unnecessary conflicts with the ongoing
    process to convert these constants to GIT_MAX_HEXSZ

  - the interface for curl_errorstr is dictated by curl
Signed-off-by: NJeff King <peff@peff.net>

There's no need to use the magic "100" when a strbuf can do
it for us.
Signed-off-by: NJeff King <peff@peff.net>

We'd prefer to avoid unchecked snprintf calls because
truncation can lead to unexpected results.

These are all cases where truncation shouldn't ever happen,
because the input to snprintf is fixed in size. That makes
them candidates for xsnprintf(), but it's simpler still to
just use the heap, and then nobody has to wonder if "100" is
big enough.

We'll use xstrfmt() where possible, and a strbuf when we need
the resulting size or to reuse the same buffer in a loop.
Signed-off-by: NJeff King <peff@peff.net>

After receive-pack reads the pack header from the client, it
feeds the already-read part to index-pack and unpack-objects
via their --pack-header command-line options.  To do so, we
format it into a fixed buffer, then duplicate it into the
child's argv_array.

Our buffer is long enough to handle any possible input, so
this isn't wrong. But it's more complicated than it needs to
be; we can just argv_array_pushf() the final value and avoid
the intermediate copy. This drops the magic number and is
more efficient, too.

Note that we need to push to the argv_array in order, which
means we can't do the push until we are in the "unpack-objects
versus index-pack" conditional.  Rather than duplicate the
slightly complicated format specifier, I pushed it into a
helper function.
Signed-off-by: NJeff King <peff@peff.net>

When name-rev needs to format an actual name, we do so into
a fixed-size buffer. That includes the actual ref tip, as
well as any traversal information. Since refs can exceed
1024 bytes, this means you can get a bogus result. E.g.,
doing:

   git tag $(perl -e 'print join("/", 1..1024)')
   git describe --contains HEAD^

results in ".../282/283", when it should be
".../1023/1024~1".

We can solve this by using a heap buffer. We'll use a
strbuf, which lets us write into the same buffer from our
loop without having to reallocate.
Signed-off-by: NJeff King <peff@peff.net>

We generate a reflog message that contains some fixed text
plus a branch name, and use a buffer of size PATH_MAX + 20.
This mostly works if you assume that refnames are shorter
than PATH_MAX, but:

  1. That's not necessarily true. PATH_MAX is not always the
     filesystem's limit.

  2. The "20" is not sufficiently large for the fixed text
     anyway.

Let's just switch to a heap buffer so we don't have to even
care.
Signed-off-by: NJeff King <peff@peff.net>

In create_branch() we write the reflog msg into a buffer in
the main function, but then use it only inside a
conditional. If you carefully follow the logic, you can
confirm that we never use the buffer uninitialized nor write
when it would not be used. But we can make this a lot more
obvious by simply moving the write step inside the
conditional.
Signed-off-by: NJeff King <peff@peff.net>

Like the previous commit, we'd like to avoid the assumption
that refs fit into PATH_MAX-sized buffers. These callsites
have an extra twist, though: they write the refnames using
mksnpath. This does two things beyond a regular snprintf:

  1. It quietly writes "/bad-path/" when truncation occurs.
     This saves the caller having to check the error code,
     but if you aren't actually feeding the result to a
     system call (and we aren't here), it's questionable.

  2. It calls cleanup_path(), which removes leading
     instances of "./".  That's questionable when dealing
     with refnames, as we could silently canonicalize a
     syntactically bogus refname into a valid one.

Let's convert each case to use a strbuf. This is preferable
to xstrfmt() because we can reuse the same buffer as we
loop.
Signed-off-by: NJeff King <peff@peff.net>

Many functions which handle refs use a PATH_MAX-sized buffer
to do so. This is mostly reasonable as we have to write
loose refs into the filesystem, and at least on Linux the 4K
PATH_MAX is big enough that nobody would care. But:

  1. The static PATH_MAX is not always the filesystem limit.

  2. On other platforms, PATH_MAX may be much smaller.

  3. As we move to alternate ref storage, we won't be bound
     by filesystem limits.

Let's convert these to heap buffers so we don't have to
worry about truncation or size limits.

We may want to eventually constrain ref lengths for sanity
and to prevent malicious names, but we should do so
consistently across all platforms, and in a central place
(like the ref code).
Signed-off-by: NJeff King <peff@peff.net>

Part of the reflog content comes from the environment, which
can be much larger than our fixed buffer. Let's use a heap
buffer so we avoid truncating it.
Signed-off-by: NJeff King <peff@peff.net>

We format the tag header into a fixed 1024-byte buffer. But
since the tag-name and tagger ident can be arbitrarily
large, we may unceremoniously die with "tag header too big".
Let's just use a strbuf instead.

Note that it looks at first glance like we can just format
this directly into the "buf" strbuf where it will ultimately
go. But that buffer may already contain the tag message, and
we have no easy way to prepend formatted data to a strbuf
(we can only splice in an already-generated buffer). This
isn't a performance-critical path, so going through an extra
buffer isn't a big deal.
Signed-off-by: NJeff King <peff@peff.net>

To generate a patch id, we format the diff header into a
fixed-size buffer, and then feed the result to our sha1
computation. The fixed buffer has size '4*PATH_MAX + 20',
which in theory accommodates the four filenames plus some
extra data. Except:

  1. The filenames may not be constrained to PATH_MAX. The
     static value may not be a real limit on the current
     filesystem. Moreover, we may compute patch-ids for
     names stored only in git, without touching the current
     filesystem at all.

  2. The 20 bytes is not nearly enough to cover the
     extra content we put in the buffer.

As a result, the data we feed to the sha1 computation may be
truncated, and it's possible that a commit with a very long
filename could erroneously collide in the patch-id space
with another commit. For instance, if one commit modified
"really-long-filename/foo" and another modified "bar" in the
same directory.

In practice this is unlikely. Because the filenames are
repeated, and because there's a single cutoff at the end of
the buffer, the offending filename would have to be on the
order of four times larger than PATH_MAX.

We could fix this by moving to a strbuf. However, we can
observe that the purpose of formatting this in the first
place is to feed it to git_SHA1_Update(). So instead, let's
just feed each part of the formatted string directly. This
actually ends up more readable, and we can even factor out
some duplicated bits from the various conditional branches.

Technically this may change the output of patch-id for very
long filenames, but it's not worth making an exception for
this in the --stable output. It was a bug, and one that only
affected an unlikely set of paths.  And anyway, the exact
value would have varied from platform to platform depending
on the value of PATH_MAX, so there is no "stable" value.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since git_path_buf() is smart enough to replace "objects/"
with the correct object path, we can use it instead of
manually assembling the path. That's slightly shorter, and
will clean up any non-canonical bits in the path.
Signed-off-by: NJeff King <peff@peff.net>

The odb_mkstemp() function expects the caller to provide a
fixed buffer to write the resulting tempfile name into. But
it creates the template using snprintf without checking the
return value. This means we could silently truncate the
filename.

In practice, it's unlikely that the truncation would end in
the template-pattern that mkstemp needs to open the file. So
we'd probably end up failing either way, unless the path was
specially crafted.

The simplest fix would be to notice the truncation and die.
However, we can observe that most callers immediately
xstrdup() the result anyway. So instead, let's switch to
using a strbuf, which is easier for them (and isn't a big
deal for the other 2 callers, who can just strbuf_release
when they're done with it).

Note that many of the callers used static buffers, but this
was purely to avoid putting a large buffer on the stack. We
never passed the static buffers out of the function, so
there's no complicated memory handling we need to change.
Signed-off-by: NJeff King <peff@peff.net>

The odb_mkstemp function does not return an error; it dies
on failure instead. But many of its callers compare the
resulting descriptor against -1 and die themselves.

Mostly this is just pointless, but it does raise a question
when looking at the callers: if they show the results of the
"template" buffer after a failure, what's in it? The answer
is: it doesn't matter, because it cannot happen.

So let's make that clear by removing the bogus error checks.
In bitmap_writer_finish(), we can drop the error-handling
code entirely. In the other two cases, it's shared with the
open() in another code path; we can just move the
error-check next to that open() call.

And while we're at it, let's flesh out the function's
docstring a bit to make the error behavior clear.
Signed-off-by: NJeff King <peff@peff.net>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

sha1dc/sha1.c wanted to check the endianness of the target platform
at compilation time and used a CPP macro with a rather overly
generic name, "BIGENDIAN", to pass the result of the check around
in the file.  It wasn't prepared for the same macro set to 0
(false) by the platform to signal that the target is _not_ a big
endian box, and assumed that the endianness detection logic it has
alone would be the one that is setting the macro, resulting in a
breakage on Windows.  This has been fixed by using a bit less
generic name for the same purpose.

* jk/sha1dc:
  sha1dc: avoid CPP macro collisions

The name-hash used for detecting paths that are different only in
cases (which matter on case insensitive filesystems) has been
optimized to take advantage of multi-threading when it makes sense.

* jh/memihash-opt:
  name-hash: add test-lazy-init-name-hash to .gitignore
  name-hash: add perf test for lazy_init_name_hash
  name-hash: add test-lazy-init-name-hash
  name-hash: perf improvement for lazy_init_name_hash
  hashmap: document memihash_cont, hashmap_disallow_rehash api
  hashmap: add disallow_rehash setting
  hashmap: allow memihash computation to be continued
  name-hash: specify initial size for istate.dir_hash table

Code clean-up.

* jk/fast-import-cleanup:
  pack.h: define largest possible encoded object size
  encode_in_pack_object_header: respect output buffer length
  fast-import: use xsnprintf for formatting headers
  fast-import: use xsnprintf for writing sha1s

Code cleanup.

* sg/skip-prefix-in-prettify-refname:
  refs.c: use skip_prefix() in prettify_refname()

Doc update.

* ab/branch-list-doc:
  branch doc: update description for `--list`
  branch doc: change `git branch <pattern>` to use `<branchname>`

Code clean-up.

* jk/pager-in-use:
  pager_in_use: use git_env_bool()

Recent enhancement to "git stash push" command to support pathspec
to allow only a subset of working tree changes to be stashed away
was found to be too chatty and exposed the internal implementation
detail (e.g. when it uses reset to match the index to HEAD before
doing other things, output from reset seeped out).  These, and
other chattyness has been fixed.

* tg/stash-push-fixup:
  stash: keep untracked files intact in stash -k
  stash: pass the pathspec argument to git reset
  stash: don't show internal implementation details

"git checkout" is taught the "--recurse-submodules" option.

* sb/checkout-recurse-submodules:
  builtin/read-tree: add --recurse-submodules switch
  builtin/checkout: add --recurse-submodules switch
  entry.c: create submodules when interesting
  unpack-trees: check if we can perform the operation for submodules
  unpack-trees: pass old oid to verify_clean_submodule
  update submodules: add submodule_move_head
  submodule.c: get_super_prefix_or_empty
  update submodules: move up prepare_submodule_repo_env
  submodules: introduce check to see whether to touch a submodule
  update submodules: add a config option to determine if submodules are updated
  update submodules: add submodule config parsing
  make is_submodule_populated gently
  lib-submodule-update.sh: define tests for recursing into submodules
  lib-submodule-update.sh: replace sha1 by hash
  lib-submodule-update: teach test_submodule_content the -C <dir> flag
  lib-submodule-update.sh: do not use ./. as submodule remote
  lib-submodule-update.sh: reorder create_lib_submodule_repo
  submodule--helper.c: remove duplicate code
  connect_work_tree_and_git_dir: safely create leading directories

Build fix for NO_PTHREADS build.

* bw/grep-recurse-submodules:
  grep: fix builds with with no thread support
  grep: set default output method

Doc update.

* km/config-grammofix:
  doc/config: grammar fixes for core.{editor,commentChar}

A test retitling.

* sb/t3600-rephrase:
  t3600: rename test to describe its functionality

A test fix.

* sb/submodule-update-initial-runs-custom-script:
  t7406: correct test case for submodule-update initial population

A test fix.

* jk/quote-env-path-list-component:
  t5615: fix a here-doc syntax error

Code clean-up.

* rs/update-hook-optim:
  receive-pack: simplify run_update_post_hook()

Code clean-up.

* rs/shortlog-cleanup:
  shortlog: don't set after_subject to an empty string

Code clean-up.

* rs/path-name-safety-cleanup:
  revision: remove declaration of path_name()

Code clean-up.

* rs/http-push-cleanup:
  http-push: don't check return value of lookup_unknown_object()

Code clean-up.

* sb/wt-status-cleanup:
  wt-status: simplify by using for_each_string_list_item

Code clean-up.

* jk/pack-name-cleanups:
  index-pack: make pointer-alias fallbacks safer
  replace snprintf with odb_pack_name()
  odb_pack_keep(): stop generating keepfile name
  sha1_file.c: make pack-name helper globally accessible
  move odb_* declarations out of git-compat-util.h

Code clean-up.

* jk/rev-parse-cleanup:
  rev-parse: simplify parsing of ref options
  rev-parse: add helper for parsing "--foo/--foo="
  rev-parse: use skip_prefix when parsing options

Code clean-up.

* rs/blame-code-cleanup:
  blame: move blame_entry duplication to add_blame_entry()

A few unterminated here documents in tests were fixed, which in
turn revealed incorrect expectations the tests make. These tests
have been updated.

* st/verify-tag:
  t7004, t7030: fix here-doc syntax errors