The new call will read from a file descriptor into a strbuf once. The
underlying call xread is just run once. xread only reattempts
reading in case of EINTR, which makes it suitable to use for a
nonblocking read.
Signed-off-by: NStefan Beller <sbeller@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This function is also used in other builtins than stripspace, so it
makes sense to have it in a more generic place.  Since it operates
on an strbuf and the function is declared in strbuf.h, move it to
strbuf.c and add the corresponding prefix to its name, just like
other API functions in the strbuf_* family.

Also switch all current users of stripspace() to the new function
name and keep a temporary wrapper inline function for any topic
branches still using stripspace().
Reviewed-by: NMatthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: NTobias Klauser <tklauser@distanz.ch>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We sometimes sprintf into fixed-size buffers when we know
that the buffer is large enough to fit the input (either
because it's a constant, or because it's numeric input that
is bounded in size). Likewise with strcpy of constant
strings.

However, these sites make it hard to audit sprintf and
strcpy calls for buffer overflows, as a reader has to
cross-reference the size of the array with the input. Let's
use xsnprintf instead, which communicates to a reader that
we don't expect this to overflow (and catches the mistake in
case we do).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The sha1_to_hex and find_unique_abbrev functions always
write into reusable static buffers. There are a few problems
with this:

  - future calls overwrite our result. This is especially
    annoying with find_unique_abbrev, which does not have a
    ring of buffers, so you cannot even printf() a result
    that has two abbreviated sha1s.

  - if you want to put the result into another buffer, we
    often strcpy, which looks suspicious when auditing for
    overflows.

This patch introduces sha1_to_hex_r and find_unique_abbrev_r,
which write into a user-provided buffer. Of course this is
just punting on the overflow-auditing, as the buffer
obviously needs to be GIT_SHA1_HEXSZ + 1 bytes. But it is
much easier to audit, since that is a well-known size.

We retain the non-reentrant forms, which just become thin
wrappers around the reentrant ones. This patch also adds a
strbuf variant of find_unique_abbrev, which will be handy in
later patches.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The loop in strbuf_read() uses xread() repeatedly while extending
the strbuf until the call returns zero.  If the buffer is
sufficiently large to begin with, this results in xread()
returning the remainder of the file to the end (returning
non-zero), the loop extending the strbuf, and then making another
call to xread() to have it return zero.

By using read_in_full(), we can tell when the read reached the end
of file: when it returns less than was requested, it's eof.  This
way we can avoid an extra iteration that allocates an extra 8kB
that is never used.
Signed-off-by: NJim Hill <gjthill@gmail.com>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The return value of strftime is poorly designed; when it
returns 0, the caller cannot tell if the buffer was not
large enough, or if the output was actually 0 bytes. In the
original implementation of strbuf_addftime, we simply punted
and guessed that our 128-byte hint would be large enough.

We can do better, though, if we're willing to treat strftime
like less of a black box. We can munge the incoming format
to make sure that it never produces 0-length output, and
then "fix" the resulting output.  That lets us reliably grow
the buffer based on strftime's return value.
Clever-idea-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

It is currently declared to return int, which could overflow for
large files.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This feeds the format directly to strftime. Besides being a
little more flexible, the main advantage is that your system
strftime may know more about your locale's preferred format
(e.g., how to spell the days of the week).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.

The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

to:

  real    0m6.768s
  user    0m6.340s
  sys     0m0.432s

for a wall-clock speedup of 21%.

Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

As with the recent speedup to strbuf_addch, we can avoid
calling strbuf_grow() in a tight loop of single-character
adds by instead checking strbuf_avail.

Note that we would instead call strbuf_addch directly here,
but it does more work than necessary: it will NUL-terminate
the result for each character read. Instead, in this loop we
read the characters one by one and then add the terminator
manually at the end.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m10.948s
  user    0m10.548s
  sys     0m0.412s

to:

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

for a wall-clock speedup of 21%.
Helped-by: NEric Sunshine <sunshine@sunshineco.com>
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

strbuf_getwholeline calls getc in a tight loop. On modern
libc implementations, the stdio code locks the handle for
every operation, which means we are paying a significant
overhead.  We can get around this by locking the handle for
the whole loop and using the unlocked variant.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

to:

  real    0m10.953s
  user    0m10.384s
  sys     0m0.580s

for a wall-clock speedup of 42%. All times are best-of-3,
and done on a glibc 2.19 system.

Note that we call into strbuf_grow while holding the lock.
It's possible for that function to call other stdio
functions (e.g., printing to stderr when dying due to malloc
error); however, the POSIX.1-2001 definition of flockfile
makes it clear that the locks are per-handle, so we are fine
unless somebody else tries to read from our same handle.
This doesn't ever happen in the current code, and is
unlikely to be added in the future (we would have to do
something exotic like add a die_routine that tried to read
from stdin).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

strbuf_getwholeline calls fgetc in a tight loop. Using the
getc form, which can be implemented as a macro, should be
faster (and we do not care about it evaluating our argument
twice, as we just have a plain variable).

On my glibc system, running "git rev-parse
refs/heads/does-not-exist" on a file with an extremely large
(1.6GB) packed-refs file went from (best of 3 runs):

  real    0m19.383s
  user    0m18.876s
  sys     0m0.528s

to:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

for a wall-clock speedup of 2.5%.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The strbuf_add_commented_lines() function passes a pair of prefixes,
one to be used for a non-empty line, and the other for an empty
line, to underlying add_lines().  The former is set to a comment
char followed by a SP, while the latter is set to just the comment
char.  This is designed to give a SP after the comment character,
e.g. "# <user text>\n", on a line with some text, and to avoid
emitting an unsightly "# \n" for an empty line.

Teach this machinery to also use the latter space-less prefix when
the payload line begins with a tab, to show e.g. "#\t<user text>\n";
otherwise we will end up showing "# \t<user text>\n" which is
similarly unsightly.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move strbuf_addchars() to strbuf.c, where it belongs, and make it
available for other callers.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Move most of the code of absolute_path() into the new function
strbuf_add_absolute_path() and in the process transform it to use
struct strbuf and xgetcwd() instead of a PATH_MAX-sized buffer,
which can be too small on some file systems.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add strbuf_getcwd(), which puts the current working directory into a
strbuf.  Because it doesn't use a fixed-size buffer it supports
arbitrarily long paths, provided the platform's getcwd() does as well.
At least on Linux and FreeBSD it handles paths longer than PATH_MAX
just fine.
Suggested-by: NKarsten Blees <karsten.blees@gmail.com>
Helped-by: NDuy Nguyen <pclouds@gmail.com>
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The ends_with function is essentially a simplified version
of strip_suffix, in which we throw away the stripped length.
Implementing it as an inline on top of strip_suffix has two
advantages:

  1. We save a bit of duplicated code.

  2. The suffix is typically a string literal, and we call
     strlen on it. By making the function inline, many
     compilers can replace the strlen call with a constant.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

You can use a strbuf to build up a string from parts, and
then detach it. In the general case, you might use multiple
strbuf_add* functions to do the building. However, in many
cases, a single strbuf_addf is sufficient, and we end up
with:

  struct strbuf buf = STRBUF_INIT;
  ...
  strbuf_addf(&buf, fmt, some, args);
  str = strbuf_detach(&buf, NULL);

We can make this much more readable (and avoid introducing
an extra variable, which can clutter the code) by
introducing a convenience function:

  str = xstrfmt(fmt, some, args);
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is a convenience wrapper around `reencode_string_len`
and `strbuf_attach`.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is a convenience wrapper to call tolower on each
character of the string.

This makes config's lowercase() function obsolete, though
note that because we have a strbuf, we are careful to
operate over the whole strbuf, rather than assuming that a
NUL is the end-of-string.

We could continue to offer a pure-string lowercase, but
there would be no callers (in most pure-string cases, we
actually duplicate and lowercase the duplicate, for which we
have the xstrdup_tolower wrapper).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We have two implementations of the same function; let's drop
that to one. We take the name from daemon.c, but the
implementation (which is just slightly more efficient) from
the config code.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

strbuf_trim() strips whitespace from the end, then the beginning of
a strbuf.  Those operations are duplicated in strbuf_rtrim() and
strbuf_ltrim().

Replace strbuf_trim() implementation with calls to strbuf_rtrim(),
then strbuf_ltrim().
Signed-off-by: NBrian Gesiak <modocache@gmail.com>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

As starts_with() and ends_with() have been used to
replace prefixcmp() and suffixcmp() respectively,
we can now remove them.
Signed-off-by: NChristian Couder <chriscool@tuxfamily.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

prefixcmp() and suffixcmp() share the common "cmp" suffix that
typically are used to name functions that can be used for ordering,
but they can't, because they are not antisymmetric:

        prefixcmp("foo", "foobar") < 0
        prefixcmp("foobar", "foo") == 0

We in fact do not use these functions for ordering.  Replace them
with functions that just check for equality.

Add starts_with() and end_with() that will be used to replace
prefixcmp() and suffixcmp(), respectively, as the first step.  These
are named after corresponding functions/methods in programming
languages, like Java, Python and Ruby.

In vcs-svn/fast_export.c, there was already an ends_with() function
that did the same thing. Let's use the new one instead while at it.
Signed-off-by: NChristian Couder <chriscool@tuxfamily.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Humanization of downloaded size is done in the same function as text
formatting in 'process.c'. The code cannot be reused easily elsewhere.

Separate text formatting from size simplification and make the
function public in strbuf so that it can easily be used by other
callers.

We now can use strbuf_humanise_bytes() for both downloaded size and
download speed calculation. One of the drawbacks is that speed will
now look like this when download is stalled: "0 bytes/s" instead of
"0 KiB/s".
Signed-off-by: NAntoine Pelisse <apelisse@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Some users do want to write a line that begin with a pound sign, #,
in their commit log message.  Many tracking system recognise
a token of #<bugid> form, for example.

The support we offer these use cases is not very friendly to the end
users.  They have a choice between

 - Don't do it.  Avoid such a line by rewrapping or indenting; and

 - Use --cleanup=whitespace but remove all the hint lines we add.

Give them a way to set a custom comment char, e.g.

    $ git -c core.commentchar="%" commit

so that they do not have to do either of the two workarounds.

[jc: although I started the topic, all the tests and documentation
updates, many of the call sites of the new strbuf_add_commented_*()
functions, and the change to git-submodule.sh scripted Porcelain are
from Ralf.]
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Signed-off-by: NRalf Thielow <ralf.thielow@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Substantially the same code is present in http-push.c and imap-send.c,
so make a library function out of it.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The word "delimiter" suggests that the argument separates the
substrings, whereas in fact (1) the delimiter characters are included
in the output, and (2) if the input string ends with the delimiter,
then the output does not include a final empty string.  So rename the
"delim" arguments of the strbuf_split() family of functions to
"terminator", which is more suggestive of how it is used.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

While iterating, update str and slen to keep track of the part of the
string that hasn't been processed yet rather than computing things
relative to the start of the original string.  This eliminates one
local variable, reduces the scope of another, and reduces the amount
of arithmetic needed within the loop.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

Use ALLOC_GROW() rather than inline code to manage memory in
strbuf_split_buf().  Rename "pos" to "nr" because it better describes
the use of the variable and it better conforms to the "ALLOC_GROW"
idiom.

Also, instead of adding a sentinal NULL value after each entry is
added to the list, only add it once after all of the entries have been
added.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJeff King <peff@peff.net>

The current behavior is to return NULL when strbuf did not
actually allocate a string. This can be quite surprising to
callers, though, who may feed the strbuf from arbitrary data
and expect to always get a valid value.

In most cases, it does not make a difference because calling
any strbuf function will cause an allocation (even if the
function ends up not inserting any data). But if the code is
structured like:

  struct strbuf buf = STRBUF_INIT;
  if (some_condition)
	  strbuf_addstr(&buf, some_string);
  return strbuf_detach(&buf, NULL);

then you may or may not return NULL, depending on the
condition. This can cause us to segfault in http-push
(when fed an empty URL) and in http-backend (when an empty
parameter like "foo=bar&&" is in the $QUERY_STRING).

This patch forces strbuf_detach to allocate an empty
NUL-terminated string when it is called on a strbuf that has
not been allocated.

I investigated all call-sites of strbuf_detach. The majority
are either not affected by the change (because they call a
strbuf_* function unconditionally), or can handle the empty
string just as easily as NULL.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Also remove an unused function.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

These functions are helpful when we do not want to expose \n to
translators. For example

    printf("hello world\n");

can be converted to

    printf_ln(_("hello world"));
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The comment even said that it should eventually go there.  While at
it, match the calling convention and name of the function to the
strbuf_get*line family.  So it now is strbuf_getwholeline_fd.
Signed-off-by: NThomas Rast <trast@student.ethz.ch>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This just follows the rfc3986 rules for percent-encoding
url data into a strbuf.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When a contributor asks the integrator to merge her history, a signed tag
can be a good vehicle to communicate the authenticity of the request while
conveying other information such as the purpose of the topic.

E.g. a signed tag "for-linus" can be created, and the integrator can run:

   $ git pull git://example.com/work.git/ for-linus

This would allow the integrator to run "git verify-tag FETCH_HEAD" to
validate the signed tag.

Update fmt-merge-msg so that it pre-fills the merge message template with
the body (but not signature) of the tag object to help the integrator write
a better merge message, in the same spirit as the existing merge.log summary
lines.

The message that comes from GPG signature validation is also included in
the merge message template to help the integrator verify it, but they are
prefixed with "#" to make them comments.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This use of strbuf_grow() is a historical artifact that was once used to
ensure that strbuf.buf was allocated and properly nul-terminated.  This
was added before the introduction of the slopbuf in b315c5c0, which
guarantees that strbuf.buf always points to a usable nul-terminated string.
So let's remove it.
Signed-off-by: NBrandon Casey <drafnel@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

In the case where sb is initialized to the slopbuf (through
strbuf_init(sb,0) or STRBUF_INIT), strbuf_grow() loses the terminating
nul: it grows the buffer, but gives ALLOC_GROW a NULL source to avoid
it being freed.  So ALLOC_GROW does not copy anything to the new
memory area.

This subtly broke the call to strbuf_getline in read_next_command()
[fast-import.c:1855], which goes

    strbuf_detach(&command_buf, NULL);  # command_buf is now = STRBUF_INIT
    stdin_eof = strbuf_getline(&command_buf, stdin, '\n');
    if (stdin_eof)
            return EOF;

In strbuf_getwholeline, this did

    strbuf_grow(sb, 0);  # loses nul-termination
    if (feof(fp))
            return EOF;
    strbuf_reset(sb);    # this would have nul-terminated!

Valgrind found this because fast-import subsequently uses prefixcmp()
on command_buf.buf, which after the EOF exit contains only
uninitialized memory.

Arguably strbuf_getwholeline is also broken, in that it touches the
buffer before deciding whether to do any work.  However, it seems more
futureproof to not let the strbuf API lose the nul-termination by its
own fault.

So make sure that strbuf_grow() puts in a nul even if it has nowhere
to copy it from.  This makes strbuf_grow(sb, 0) a semantic no-op as
far as readers of the buffer are concerned.

Also remove the nul-termination added by strbuf_init, which is made
redudant.
Signed-off-by: NThomas Rast <trast@student.ethz.ch>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The strbuf_split function takes a strbuf as input, and
outputs a list of strbufs. However, there is no reason that
the input has to be a strbuf, and not an arbitrary buffer.

This patch adds strbuf_split_buf for a length-delimited
buffer, and strbuf_split_str for NUL-terminated strings.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Sometimes when splitting, you only want a limited number of
fields, and for the final field to contain "everything
else", even if it includes the delimiter.

This patch introduces strbuf_split_max, which provides a
"max number of fields" parameter; it behaves similarly to
perl's "split" with a 3rd field.

The existing 2-argument form of strbuf_split is retained for
compatibility and ease-of-use.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>