It was stupid to link the same element twice to lock_file_list
and end up in a loop, so we certainly need a fix.

But it is not like we are taking a lock on multiple files in
this case.  It is just that we leave the linked element on the
list even after commit_lock_file() successfully removes the
cruft.

We cannot remove the list element in commit_lock_file(); if we
are interrupted in the middle of list manipulation, the call to
remove_lock_file_on_signal() will happen with a broken list
structure pointed by lock_file_list, which would cause the cruft
to remain, so not removing the list element is the right thing
to do.  Instead we should be reusing the element already on the
list.

There is already a code for that in lock_file() function in
lockfile.c.  The code checks lk->next and the element is linked
only when it is not already on the list -- which is incorrect
for the last element on the list (which has NULL in its next
field), but if you read the check as "is this element already on
the list?" it actually makes sense.  We do not want to link it
on the list again, nor we would want to set up signal/atexit
over and over.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

In some cases we did not even bother to check the return value of
mmap() and just assume it worked.  This is bad, because if we are
out of virtual address space the kernel returned MAP_FAILED and we
would attempt to dereference that address, segfaulting without any
real error output to the user.

We are replacing all calls to mmap() with xmmap() and moving all
MAP_FAILED checking into that single location.  If a mmap call
fails we try to release enough least-recently-used pack windows
to possibly succeed, then retry the mmap() attempt.  If we cannot
mmap even after releasing pack memory then we die() as none of our
callers have any reasonable recovery strategy for a failed mmap.
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Not using reflog for tags/ was very sensible; not giving reflog
for the remotes/ was not.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When ref@{N} is specified on a ref that has only M entries (M < N),
instead of saying the initial timestamp the reflog has, warn that
there is only M entries.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a mechanical clean-up of the way *.c files include
system header files.

 (1) sources under compat/, platform sha-1 implementations, and
     xdelta code are exempt from the following rules;

 (2) the first #include must be "git-compat-util.h" or one of
     our own header file that includes it first (e.g. config.h,
     builtin.h, pkt-line.h);

 (3) system headers that are included in "git-compat-util.h"
     need not be included in individual C source files.

 (4) "git-compat-util.h" does not have to include subsystem
     specific header files (e.g. expat.h).
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When renaming a branch, the corresponding config section should
be renamed, too.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

"git push $URL HEAD~6" created a bogus ref HEAD~6 immediately
under $GIT_DIR of the remote repository.  While we should keep
refspecs that have arbitrary extended SHA-1 expression on the
source side working (e.g. "HEAD~6:refs/tags/yesterday"), we
should not create bogus ref on the other end.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This changes the signature of rename_ref() in refs.[hc] to include a
logmessage for the reflogs.

Also, builtin-branch.c is modified to provide a proper logmessage + call
setup_ident() before any logmessages are written.
Signed-off-by: NLars Hjemli <hjemli@gmail.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The current check for symlinked reflogs was based on stat(2), which is
utterly embarrassing.

Fix it, and add a matching testcase.
Signed-off-by: NLars Hjemli <hjemli@gmail.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Extend git-branch with the following options:

  git-branch -m|-M [<oldbranch>] newbranch

The -M variation is required to force renaming over an exsisting
branchname.

This also indroduces $GIT_DIR/RENAME_REF which is a "metabranch"
used when renaming branches. It will always hold the original sha1
for the latest renamed branch.

Additionally, if $GIT_DIR/logs/RENAME_REF exists, all branch rename
events are logged there.

Finally, some testcases are added to verify the new options.
Signed-off-by: NLars Hjemli <hjemli@gmail.com>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This fixes the previous implementation which failed to optimize
repositories with tons of lightweight tags.  The updated
packed-refs format begins with "# packed-refs with:" line that
lists the kind of extended data the file records.  Currently,
there is only one such extension defined, "peeled".  This stores
the "peeled tag" on a line that immediately follows a line for a
tag object itself in the format "^<sha-1>".

The header line itself and any extended data are ignored by
older implementation, so packed-refs file generated with this
version can still be used by older git.  packed-refs made by
older git can of course be used with this version.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This would speed up "show-ref -d" in a repository with mostly
packed tags.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

An earlier commit b37a562a added a check to see if the ref
points at a valid object (as a part of 'negative ref' support
which we currently do not use), but did so only while iterating
over both packed and loose refs, and forgot to apply the same
check while iterating over the remaining ones.

We might want to replace the "if null then omit it" check with
"eh --- what business does a 0{40} value have here?" complaint
later since we currently do not use negative refs, but that is
a separate issue.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

After deleting a branch l/k, you should be able to create a
branch l.  Earlier we added remove_empty_directories() on the
ref creation side to remove leftover .git/refs/l directory but
we also need a matching code to remove .git/logs/refs/l
directory.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

It used to mean "create log file for any ref that is updated",
but now it creates new log files only for branch heads.

The old behaviour made this configuration less useful than
otherwise it would be; automatically creating log file for tags
is almost always not useful.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Often I find myself wanting to say 'tip of "next" before I
merged the last three topics'.  Now I can say that with:

	git log next@{3}..next

Since small integers alone are invalid input strings to
approxidate, there is no fear of confusion.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

lock_ref_sha1_basic relies on errno beeing set to EISDIR by the
call to read() in resolve_ref() to detect directories.  But calling
read() on a directory under NetBSD returns EPERM, and even succeeds
for local filesystems on FreeBSD.
Signed-off-by: NDennis Stosberg <dennis@stosberg.net>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

lock_ref_sha1_basic relies on errno beeing set to EISDIR by the
call to read() in resolve_ref() to detect directories.  But calling
read() on a directory under NetBSD returns EPERM, and even succeeds
for local filesystems on FreeBSD.
Signed-off-by: NDennis Stosberg <dennis@stosberg.net>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The function repack_without_ref() passes a lock-file structure
on the stack to hold_lock_file_for_update(), which in turn
registers it to be cleaned up via atexit().  This is a big
no-no.

This is the same bug James Bottomley fixed with commit
31f584c2.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NChristian Couder <chriscool@tuxfamily.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This implements deletion of a packed ref.  Since it is a very
rare event to delete a ref compared to looking up, creating and
updating, this opts to remove the ref from the packed-ref file
instead of doing any of the filesystem based "negative ref" trick
to optimize the deletion path.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This makes the ref locking codepath to notice if an existing ref
overlaps with the ref we are creating.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This cleans up the error path in the function so it does not
die() itself sometimes while signalling an error with NULL some
other times which was inconsistent and confusing.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Once we read packed and loose refs, for_each_ref() and friends
kept using them even after write_ref_sha1() and delete_ref()
changed the refs.  This adds invalidate_cached_refs() as a way
to flush the cached information.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

It is normal to have .git/refs/heads/foo directory which is
empty after the last branch whose name starts with foo/ is
removed.  Make sure we notice this case and allow creation of
branch foo by removing the empty directory.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This adds -d flag to update-ref to allow safe deletion of ref.
Before deleting it, the command checks if the given <oldvalue>
still matches the value the caller thought the ref contained.

Similarly, it also accepts 0{40} or an empty string as <oldvalue>
to allow safe creation of a new ref.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This drops "mustexist" parameter lock_ref_sha1() and lock_any_ref_forupdate()
functions take.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

There is a format string vulnerability introduced with the packed refs
file format.
Signed-off-by: NPetr Baudis <pasky@suse.cz>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter.  They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a long overdue fix to the API for for_each_ref() family
of functions.  It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.

The updated for_each_ref() family takes a function of type

	int (*fn)(const char *, const unsigned char *, void *)

and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.

The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Current git#next is totally broken wrt. cloning over HTTP, generating refs
at random directories. Of course it's caused by the static get_pathname()
buffer. lock_ref_sha1() stores return value of mkpath()'s get_pathname()
call, then calls lock_ref_sha1_basic() which calls git_path(ref) which
calls get_pathname() at that point returning pointer to the same buffer.
So now you are sprintf()ing a format string into itself, wow! The resulting
pathnames are really cute. (If you've been paying attention, yes, the
mere fact that a format string _could_ write over itself is very wrong
and probably exploitable here. See the other mail I've just sent.)

I've never liked how we use return values of those functions so liberally,
the "allow some random number of get_pathname() return values to work
concurrently" is absolutely horrible pit and we've already fallen in this
before IIRC. I consider it an awful coding practice, you add a call
somewhere and at some other point some distant caller of that breaks since
it reuses the same return values. Not to mention this takes quite some time
to debug.

My gut feeling tells me that there might be more of this.  I don't have
time to review the rest of the users of the refs.c functions though.
Signed-off-by: NPetr Baudis <pasky@suse.cz>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This actually "turns on" the packed ref file format, now that the
infrastructure to do so sanely exists (ie notably the change to make the
reference reading logic take refnames rather than pathnames to the loose
objects that no longer necessarily even exist).

In particular, when the ref lookup hits a refname that has no loose file
associated with it, it falls back on the packed-ref information. Also, the
ref-locking code, while still using a loose file for the locking itself
(and _creating_ a loose file for the new ref) no longer requires that the
old ref be in such an unpacked state.

Finally, this does a minimal hack to git-checkout.sh to rather than check
the ref-file directly, do a "git-rev-parse" on the "heads/$refname".
That's not really wonderful - we should rather really have a special
routine to verify the names as proper branch head names, but it is a
workable solution for now.

With this, I can literally do something like

	git pack-refs
	find .git/refs -type f -print0 | xargs -0 rm -f --

and the end result is a largely working repository (ie I've done two
commits - which creates _one_ unpacked ref file - done things like run
"gitk" and "git log" etc, and it all looks ok).

There are probably things missing, but I'm hoping that the missing things
are now of the "small and obvious" kind, and that somebody else might want
to start looking at this too. Hint hint ;)
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The old code used to totally mix up the notion of a ref-name and the path
that that ref was associated with.  That was not only horribly ugly (a
number of users got the path, and then wanted to try to turn it back into
a ref-name again), but it fundamnetally doesn't work at all once we do any
setup where a ref doesn't have a 1:1 relationship with a particular
pathname.

This fixes things up so that we use the ref-name throughout, and only
turn it into a pathname once we actually look it up in the filesystem.
That makes a lot of things much clearer and more straightforward.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

You can remove a ref that is packed two different ways: either simply
repack all the refs without that one, or create a loose ref that has the
magic all-zero SHA1.

This also adds back the test that a ref actually has the object it
points to.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This also adds some very rudimentary support for the notion of packed
refs.  HOWEVER! At this point it isn't used to actually look up a ref
yet, only for listing them (ie "for_each_ref()" and friends see the
packed refs, but none of the other single-ref lookup routines).

Note how we keep two separate lists: one for the loose refs, and one for
the packed refs we read. That's so that we can easily keep the two apart,
and read only one set or the other (and still always make sure that the
loose refs take precedence).

[ From this, it's not actually obvious why we'd keep the two separate
  lists, but it's important to have the packed refs on their own list
  later on, when I add support for looking up a single loose one.

  For that case, we will want to read _just_ the packed refs in case the
  single-ref lookup fails, yet we may end up needing the other list at
  some point in the future, so keeping them separated is important ]
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Like xmalloc and xrealloc xstrdup dies with a useful message if
the native strdup() implementation returns NULL rather than a
valid pointer.

I just tried to use xstrdup in new code and found it to be missing.
However I expected it to be present as xmalloc and xrealloc are
already commonly used throughout the code.

[jc: removed the part that deals with last_XXX, which I am
 finding more and more dubious these days.]
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Jonas noticed some places say "if (X) free(X)" which is totally
unnecessary.
Signed-off-by: NJunio C Hamano <junkio@cox.net>