In a process with multiple repositories open, packfile accessors
should be associated to a single repository and not shared globally.
Move packed_git and packed_git_mru into the_repository and adjust
callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git()
and get_packed_git_mru(). This allows us to lazily initialize these
fields without caller doing that explicitly]
Signed-off-by: NStefan Beller <sbeller@google.com>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJonathan Tan <jonathantanmy@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When fopen() returns NULL, it could be because the given path does not
exist, but it could also be some other errors and the caller has to
check. Add a wrapper so we don't have to repeat the same error check
everywhere.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Make parse_object, parse_object_or_die, and parse_object_buffer take a
pointer to struct object_id.  Remove the temporary variables inserted
earlier, since they are no longer necessary.  Transform all of the
callers using the following semantic patch:

@@
expression E1;
@@
- parse_object(E1.hash)
+ parse_object(&E1)

@@
expression E1;
@@
- parse_object(E1->hash)
+ parse_object(E1)

@@
expression E1, E2;
@@
- parse_object_or_die(E1.hash, E2)
+ parse_object_or_die(&E1, E2)

@@
expression E1, E2;
@@
- parse_object_or_die(E1->hash, E2)
+ parse_object_or_die(E1, E2)

@@
expression E1, E2, E3, E4, E5;
@@
- parse_object_buffer(E1.hash, E2, E3, E4, E5)
+ parse_object_buffer(&E1, E2, E3, E4, E5)

@@
expression E1, E2, E3, E4, E5;
@@
- parse_object_buffer(E1->hash, E2, E3, E4, E5)
+ parse_object_buffer(E1, E2, E3, E4, E5)
Signed-off-by: Nbrian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

If an error occurs when or after closing the stream we call fclose(3)
again in the error handler.  The second call can exhibit undefined
behavior, so make sure to call fclose(3) at most once.  Also avoid
calling close(2) after fd has been successfully associated with the
stream, as fclose(3) has become responsible for doing that beyond
this point.

Found with Cppcheck.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Reviewed-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Apply the semantic patch contrib/coccinelle/qsort.cocci to the code
base, replacing calls of qsort(3) with QSORT.  The resulting code is
shorter and supports empty arrays with NULL pointers.
Signed-off-by: NRene Scharfe <l.s.r@web.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

struct object is one of the major data structures dealing with object
IDs.  Convert it to use struct object_id instead of an unsigned char
array.  Convert get_object_hash to refer to the new member as well.
Signed-off-by: Nbrian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: NJeff King <peff@peff.net>

Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Nbrian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change typedef each_ref_fn to take a "const struct object_id *oid"
parameter instead of "const unsigned char *sha1".

To aid this transition, implement an adapter that can be used to wrap
old-style functions matching the old typedef, which is now called
"each_ref_sha1_fn"), and make such functions callable via the new
interface. This requires the old function and its cb_data to be
wrapped in a "struct each_ref_fn_sha1_adapter", and that object to be
used as the cb_data for an adapter function, each_ref_fn_adapter().

This is an enormous diff, but most of it consists of simple,
mechanical changes to the sites that call any of the "for_each_ref"
family of functions. Subsequent to this change, the call sites can be
rewritten one by one to use the new interface.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Nbrian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Prior to d38379ec (make update-server-info more robust, 2014-09-13),
we used a straight "fopen" to create the info/refs and
objects/info/packs files, which creates the file using mode 0666
(less the default umask).

In d38379ec, we switched to creating the file with mkstemp to get a
unique filename. But mkstemp also uses the more restrictive 0600
mode to create the file. This was an unintended side effect that we
did not want, and causes problems when the repository is served by a
different user than the one running update-server-info (it is not
readable by a dumb http server running as `www`, for example).

We can fix this by using git_mkstemp_mode and specifying 0666 to
make sure that the umask is honored.

Note that we could also say "just use core.sharedrepository", as we
do call adjust_shared_perm on the result before renaming it into
place.  But that should not be necessary as long as everybody
involved is using permissive umask to allow HTTP server to read
necessary files.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We allocate pack information in a static global list but
never clean it up. This leaks memory, and means that calling
update_server_info twice will generate a buggy file (it will
have duplicate entries).
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since "git update-server-info" may be called automatically
as part of a push or a "gc --auto", we should be robust
against two processes trying to update it simultaneously.
However, we currently use a fixed tempfile, which means that
two simultaneous writers may step on each other's toes and
end up renaming junk into place.

Let's instead switch to using a unique tempfile via mkstemp.
We do not want to use a lockfile here, because it's OK for
two writers to simultaneously update (one will "win" the
rename race, but that's OK; they should be writing the same
information).

While we're there, let's clean up a few other things:

  1. Detect write errors. Report them and abort the update
     if any are found.

  2. Free path memory rather than leaking it (and clean up
     the tempfile when necessary).

  3. Use the pathdup functions consistently rather than
     static buffers or manually calculated lengths.

This last one fixes a potential overflow of "infofile" in
update_info_packs (e.g., by putting large junk into
$GIT_OBJECT_DIRECTORY). However, this overflow was probably
not an interesting attack vector for two reasons:

  a. The attacker would need to control the environment to
     do this, in which case it was already game-over.

  b. During its setup phase, git checks that the directory
     actually exists, which means it is probably shorter
     than PATH_MAX anyway.

Because both update_info_refs and update_info_packs share
these same failings (and largely duplicate each other), this
patch factors out the improved error-checking version into a
helper function.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The correct responses to a D and a T line in .git/objects/info/packs
are the same, so combine their case arms.  In both cases we already
‘goto’ out of the switch so while at it, remove a redundant ‘break’
to avoid yet another line of code.
Signed-off-by: NRalf Thielow <ralf.thielow@googlemail.com>
Reviewed-by: Jonathan Nieder <jrnieder <at> gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Essentially; s/type* /type */ as per the coding guidelines.
Signed-off-by: NFelipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This helps to notice when something's going wrong, especially on
systems which lock open files.

I used the following criteria when selecting the code for replacement:
- it was already printing a warning for the unlink failures
- it is in a function which already printing something or is
  called from such a function
- it is in a static function, returning void and the function is only
  called from a builtin main function (cmd_)
- it is in a function which handles emergency exit (signal handlers)
- it is in a function which is obvously cleaning up the lockfiles
Signed-off-by: NAlex Riesen <raa.lkml@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A NUL byte at beginning of file, or just after a newline
would provoke an invalid buf[-1] access in a few places.

* builtin-grep.c (cmd_grep): Don't access buf[-1].
* builtin-pack-objects.c (get_object_list): Likewise.
* builtin-rev-list.c (read_revisions_from_stdin): Likewise.
* bundle.c (read_bundle_header): Likewise.
* server-info.c (read_pack_info_file): Likewise.
* transport.c (insert_packed_refs): Likewise.
Signed-off-by: NJim Meyering <meyering@redhat.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When we fail to open a temporary file to be renamed to something else,
we reported the final filename, not the temporary file we failed to
open.
Signed-off-by: NAndré Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

For compatibility reasons, "git init --shared=all" does not write
"all" into the config, but a number.  In the shared setup, you
really have to support even older clients on the _same_ repository.

But git_config_perm() did not pick up on it.

Also, "git update-server-info" failed to pick up on the shared
permissions.

This patch fixes both issues, and adds a test to prove it.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Tested-by: Nmartin f krafft <madduck@madduck.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Apparently if we are unable to parse an object update-server-info
coredumps, as it doesn't bother to check the return value of its
call to parse_object.

Instead of coredumping, skip the ref.
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter.  They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a long overdue fix to the API for for_each_ref() family
of functions.  It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.

The updated for_each_ref() family takes a function of type

	int (*fn)(const char *, const unsigned char *, void *)

and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.

The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Like xmalloc and xrealloc xstrdup dies with a useful message if
the native strdup() implementation returns NULL rather than a
valid pointer.

I just tried to use xstrdup in new code and found it to be missing.
However I expected it to be present as xmalloc and xrealloc are
already commonly used throughout the code.

[jc: removed the part that deals with last_XXX, which I am
 finding more and more dubious these days.]
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This updates the type-enumeration constants introduced to reduce
the memory footprint of "struct object" to match the type bits
already used in the packfile format, by removing the former
(i.e. TYPE_* constant macros) and using the latter (i.e. enum
object_type) throughout the code for consistency.

Eventually we can stop passing around the "type strings"
entirely, and this will help - no confusion about two different
integer enumeration.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

This shrinks "struct object" by a small amount, by getting rid of the
"struct type *" pointer and replacing it with a 3-bit bitfield instead.

In addition, we merge the bitfields and the "flags" field, which
incidentally should also remove a useless 4-byte padding from the object
when in 64-bit mode.

Now, our "struct object" is still too damn large, but it's now less
obviously bloated, and of the remaining fields, only the "util" (which is
not used by most things) is clearly something that should be eventually
discarded.

This shrinks the "git-rev-list --all" memory use by about 2.5% on the
kernel archive (and, perhaps more importantly, on the larger mozilla
archive). That may not sound like much, but I suspect it's more on a
64-bit platform.

There are other remaining inefficiencies (the parent lists, for example,
probably have horrible malloc overhead), but this was pretty obvious.

Most of the patch is just changing the comparison of the "type" pointer
from one of the constant string pointers to the appropriate new TYPE_xxx
small integer constant.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Now we allow an empty line in objects/info/packs file, recognize
that and stop complaining.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The code to fetch pack index files in deployed clients have a
bug that causes it to ignore the pack file on the last line of
objects/info/packs file, so append an empty line to work it
around.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a companion patch to e23eff8b
commit.  The same logic, the same rationale that a comparison
function that returns an int should not just compute a ptrdiff_t
and return it.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

Again, dumb transport clients are too dumb to make use of the
top objects information to make a choice among multiple packs,
so computing these lines are useless for now.  We could
resurrect them if needed later.  Also dumb transport clients
presumably can do their own approximation by downloading idx
files to see how relevant each pack is for their fetch.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This does not seem to buy us much, for the same reason as the
previous change.  Dumb clients are still too dumb.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

We tried to compute pack interdependency information in
$GIT_DIR/objects/info/packs, hoping that dumb transports would
make use of it when choosing from multiple choice, but that has
never materialized, so stop computing D lines for now.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

Store pointers to referenced objects in a variable sized array instead
of linked list.  This cuts down memory usage of utilities which use
object references; e.g., git-fsck-objects --full on the git.git
repository consumes about 2 MB of memory tracked by Massif instead of
7 MB before the change.  Object refs are still the biggest consumer of
memory (57%), but the malloc overhead for a single block instead of a
linked list is substantially smaller.
Signed-off-by: NSergey Vlasov <vsu@altlinux.ru>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

One caller of deref_tag() was not careful enough to make sure
what deref_tag() returned was not NULL (i.e. we found a tag
object that points at an object we do not have).  Fix it, and
warn about refs that point at such an incomplete tag where
needed.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This updates git-ls-remote to show SHA1 names of objects that are
referred by tags, in the "ref^{}" notation.

This would make git-findtags (without -t flag) almost trivial.

    git-peek-remote . |
    sed -ne "s:^$target	"'refs/tags/\(.*\)^{}$:\1:p'

Also Pasky could do:

    git-ls-remote --tags $remote |
    sed -ne 's:\(	refs/tags/.*\)^{}$:\1:p'

to find out what object each of the remote tags refers to, and
if he has one locally, run "git-fetch $remote tag $tagname" to
automatically catch up with the upstream tags.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Modify parse_object_cheap() to also free all the entries from the tree
data structures.
Signed-off-by: NRobert Fitzsimons <robfitz@273k.net>
Signed-off-by: NJunio C Hamano <junkio@cox.net>