Change prune_ref to delete the ref using a ref transaction. To do this we also
need to add a new flag REF_ISPRUNING that will tell the transaction that we
do not want to delete this ref from the packed refs. This flag is private to
refs.c and not exposed to external callers.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

No external callers reference lock_ref_sha1 any more so let's declare it
static.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add an err argument to _begin so that on non-fatal failures in future ref
backends we can report a nice error back to the caller.
While _begin can currently never fail for other reasons than OOM, in which
case we die() anyway, we may add other types of backends in the future.
For example, a hypothetical MySQL backend could fail in _begin with
"Can not connect to MySQL server. No route to host".
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change ref_transaction_delete() to do basic error checking and return
non-zero on error. Update all callers to check the return for
ref_transaction_delete(). There are currently no conditions in _delete that
will return error but there will be in the future. Add an err argument that
will be updated on failure.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Do basic error checking in ref_transaction_create() and make it return
non-zero on error. Update all callers to check the result of
ref_transaction_create(). There are currently no conditions in _create that
will return error but there will be in the future. Add an err argument that
will be updated on failure.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Update ref_transaction_update() do some basic error checking and return
non-zero on error. Update all callers to check ref_transaction_update() for
error. There are currently no conditions in _update that will return error but
there will be in the future. Add an err argument that will be updated on
failure. In future patches we will start doing both locking and checking
for name conflicts in _update instead of _commit at which time this function
will start returning errors for these conditions.

Also check for BUGs during update and die(BUG:...) if we are calling
_update with have_old but the old_sha1 pointer is NULL.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Since all callers now use QUIET_ON_ERR we no longer need to provide an onerr
argument any more. Remove the onerr argument from the ref_transaction_commit
signature.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from commit_packed_refs() meaningful,
which should fix

 * a bug in "git clone" where it prints strerror(errno) based on
   errno, despite errno possibly being zero and potentially having
   been clobbered by that point
 * the same kind of bug in "git pack-refs"

and prepares for repack_without_refs() to get a meaningful
error message when commit_packed_refs() fails without falling into
the same bug.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from verify_lock() meaningful, which
should almost but not completely fix

 * a bug in "git fetch"'s s_update_ref, which trusts the result of an
   errno == ENOTDIR check to detect D/F conflicts

ENOTDIR makes sense as a sign that a file was in the way of a
directory we wanted to create.  Should "git fetch" also look for
ENOTEMPTY or EEXIST to catch cases where a directory was in the way
of a file to be created?
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from log_ref_setup() meaningful,
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Update repack_without_refs to take an err argument and update it if there
is a failure. Pass the err variable from ref_transaction_commit to this
function so that callers can print a meaningful error message if _commit
fails due to this function.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from lock_file() meaningful, which should
fix

 * an existing almost-bug in lock_ref_sha1_basic where it assumes
   errno==ENOENT is meaningful and could waste some work on retries

 * an existing bug in repack_without_refs where it prints
   strerror(errno) and picks advice based on errno, despite errno
   potentially being zero and potentially having been clobbered by
   that point
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Add a strbuf argument to _commit so that we can pass an error string back to
the caller. So that we can do error logging from the caller instead of from
_commit.

Longer term plan is to first convert all callers to use onerr==QUIET_ON_ERR
and craft any log messages from the callers themselves and finally remove the
onerr argument completely.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

ref_transaction_create|delete|update has no need to modify the sha1
arguments passed to it so it should use const unsigned char* instead
of unsigned char*.

Some functions, such as fast_forward_to(), already have its old/new
sha1 arguments as consts. This function will at some point need to
use ref_transaction_update() in which case this change is required.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

We do not yet need both a rollback and a free function for transactions.
Remove ref_transaction_rollback and use ref_transaction_free instead.

At a later stage we may reintroduce a rollback function if we want to start
adding reusable transactions and similar.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

When 'git remote prune' was used to delete many refs in a repository
with many refs, a lot of time was spent checking for (now) dangling
symbolic refs pointing to the deleted ref, since warn_dangling_symref()
was once per deleted ref to check all other refs in the repository.

Avoid this using the new warn_dangling_symrefs() function which
makes one pass over all refs and checks for all the deleted refs in
one go, after they have all been deleted.
Signed-off-by: NJens Lindström <jl@opera.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When 'git remote rm' or 'git remote prune' were used in a repository
with many refs, and needed to delete many remote-tracking refs, a lot
of time was spent deleting those refs since for each deleted ref,
repack_without_refs() was called to rewrite packed-refs without just
that deleted ref.

To avoid this, call repack_without_refs() first to repack without all
the refs that will be deleted, before calling delete_ref() to delete
each one completely.  The call to repack_without_ref() in delete_ref()
then becomes a no-op, since packed-refs already won't contain any of
the deleted refs.
Signed-off-by: NJens Lindström <jl@opera.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add two new functions, reflog_exists and delete_reflog, to hide the internal
reflog implementation (that they are files under .git/logs/...) from callers.
Update checkout.c to use these functions in update_refs_for_switch instead of
building pathnames and calling out to file access functions. Update reflog.c
to use these to check if the reflog exists. Now there are still many places
in reflog.c where we are still leaking the reflog storage implementation but
this at least reduces the number of such dependencies by one. Finally
change two places in refs.c itself to use the new function to check if a ref
exists or not isntead of build-path-and-stat(). Now, this is strictly not all
that important since these are in parts of refs that are implementing the
actual file storage backend but on the other hand it will not hurt either.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is consistent with the usual nomenclature.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

It has been superseded by reference transactions.  This also means
that struct ref_update can become private.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Build out the API for dealing with a bunch of reference checks and
changes within a transaction.  Define an opaque ref_transaction type
that is managed entirely within refs.c.  Introduce functions for
beginning a transaction, adding updates to a transaction, and
committing/rolling back a transaction.

This API will soon replace update_refs().
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The old signature of update_refs() required a
(const struct ref_update **) for its updates_orig argument.  The
"const" is presumably there to promise that the function will not
modify the contents of the structures.

But this declaration does not permit the function to be called with a
(struct ref_update **), which is perfectly legitimate.  C's type
system is not powerful enough to express what we'd like.  So remove
the first "const" from the declaration.

On the other hand, the function *can* promise not to modify the
pointers within the array that is passed to it without inconveniencing
its callers.  So add a "const" that has that effect, making the final
declaration
(struct ref_update * const *).
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Given that these constants are only being used when updating
references, it is inappropriate to give them such generic names as
"DIE_ON_ERR".  So prefix their names with "UPDATE_REFS_".
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The function 'invalidate_ref_cache' was introduced in 79c7ca54 (2011-10-17,
invalidate_ref_cache(): rename function from invalidate_cached_refs())
by a rename and elevated to be publicly usable in 8be8bde7 (2011-10-17,
invalidate_ref_cache(): expose this function in the refs API)

However it is not used anymore, as 8bf90dc9 (2011-10-17, write_ref_sha1():
only invalidate the loose ref cache) and (much) later 506a760d (2013-04-22,
refs: change how packed refs are deleted) removed any calls to this
function. So it seems as if we don't need that function any more,
good bye!
Signed-off-by: NStefan Beller <stefanbeller@googlemail.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.
Signed-off-by: NBrad King <brad.king@kitware.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.
Signed-off-by: NBrad King <brad.king@kitware.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Handle simple transactions for the packed-refs file at the
packed_ref_cache level via new functions lock_packed_refs(),
commit_packed_refs(), and rollback_packed_refs().

Only allow the packed ref cache to be modified (via add_packed_ref())
while the packed refs file is locked.

Change clone to add the new references within a transaction.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The lifetime of the memory pointed to by the refname and sha1
arguments to each_ref_fn was never documented, but some callers used
to assume that it was essentially permanent.  In fact the API does
*not* guarantee that these objects live beyond a single callback
invocation.

In the current code, the lifetimes are bound together with the
lifetimes of the ref_caches.  Since these are usually long, the
callers usually got away with their sloppiness.  But even today, if a
ref_cache is invalidated the memory can be freed.  And planned changes
to reference caching, needed to eliminate race conditions, will
probably need to shorten the lifetimes of these objects.

The commits leading up to this have (hopefully) fixed all of the
callers of the for_each_ref()-like functions.  This commit does the
last step: documents what each_ref_fn callbacks can assume about
object lifetimes.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

pack-refs.c doesn't contain much code, and the code it does contain is
closely related to reference handling.  Moreover, there is some
duplication between pack_refs() and repack_without_ref().  Therefore,
merge pack-refs.c into refs.c and pack-refs.h into refs.h.

The code duplication will be addressed in future commits.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The old version was inconsistent: when a reference was
REF_KNOWS_PEELED but with a null peeled value, it returned non-zero
for the current reference but zero for other references.  Change the
behavior for non-current references to match that of current_ref,
which is what callers expect.  Document the behavior.

Current callers only call peel_ref() from within a for_each_ref-style
iteration and only for the current ref; therefore, the buggy code path
was never reached.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Document the bits that can appear in the "flags" parameter passed to
an each_ref_function and/or in the ref_entry::flag field.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

"git checkout -" is a short-hand for "git checkout @{-1}" and the
"@{nth}" notation for a negative number is to find nth previous
checkout in the reflog of the HEAD to determine the name of the
branch the user was on.  We would want to find the nth most recent
reflog entry that matches "checkout: moving from X to Y" for this.

Unfortunately, reflog is implemented as an append-only file, and the
API to iterate over its entries, for_each_reflog_ent(), reads the
file in order, giving the entries from the oldest to newer.  For the
purpose of finding nth most recent one, this API forces us to record
the last n entries in a rotating buffer and give the result out only
after we read everything.  To optimize for a common case of finding
the nth most recent one for a small value of n, we also have a side
API for_each_recent_reflog_ent() that starts reading near the end of
the file, but it still has to read the entries in the "wrong" order.
The implementation of understanding @{-1} uses this interface.

This all becomes unnecessary if we add an API to let us iterate over
reflog entries in the reverse order, from the newest to older.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

A repository may have refs that are only used for its internal
bookkeeping purposes that should not be exposed to the others that
come over the network.

Teach upload-pack to omit some refs from its initial advertisement
by paying attention to the uploadpack.hiderefs multi-valued
configuration variable.  Do the same to receive-pack via the
receive.hiderefs variable.  As a convenient short-hand, allow using
transfer.hiderefs to set the value to both of these variables.

Any ref that is under the hierarchies listed on the value of these
variable is excluded from responses to requests made by "ls-remote",
"fetch", etc. (for upload-pack) and "push" (for receive-pack).

Because these hidden refs do not count as OUR_REF, an attempt to
fetch objects at the tip of them will be rejected, and because these
refs do not get advertised, "git push :" will not see local branches
that have the same name as them as "matching" ones to be sent.

An attempt to update/delete these hidden refs with an explicit
refspec, e.g. "git push origin :refs/hidden/22", is rejected.  This
is not a new restriction.  To the pusher, it would appear that there
is no such ref, so its push request will conclude with "Now that I
sent you all the data, it is time for you to update the refs.  I saw
that the ref did not exist when I started pushing, and I want the
result to point at this commit".  The receiving end will apply the
compare-and-swap rule to this request and rejects the push with
"Well, your update request conflicts with somebody else; I see there
is such a ref.", which is the right thing to do. Otherwise a push to
a hidden ref will always be "the last one wins", which is not a good
default.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Store references hierarchically in a tree that matches the
pseudo-directory structure of the reference names.  Add a new kind of
ref_entry (with flag REF_DIR) to represent a whole subdirectory of
references.  Sort ref_dirs one subdirectory at a time.

NOTE: the dirs can now be sorted as a side-effect of other function
calls.  Therefore, it would be problematic to do something from a
each_ref_fn callback that could provoke the sorting of a directory
that is currently being iterated over (i.e., the directory containing
the entry that is being processed or any of its parents).

This is a bit far-fetched, because a directory is always sorted just
before being iterated over.  Therefore, read-only accesses cannot
trigger the sorting of a directory whose iteration has already
started.  But if a callback function would add a reference to a parent
directory of the reference in the iteration, then try to resolve a
reference under that directory, a re-sort could be triggered and cause
the iteration to work incorrectly.

Nevertheless...add a comment in refs.h warning against modifications
during iteration.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The extra_refs provided a kludgy way to create fake references at a
global level in the hope that they would only affect some particular
code path.  The last user of this API been rewritten, so strip this
stuff out before somebody else gets the bad idea of using it.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add a new function add_packed_ref() that adds a reference directly to
the in-memory packed reference cache.  This will be useful for
creating local references while cloning.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Try consistently to use the name "sha1" for parameters to which a SHA1
will be stored.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Try to consistently use the variable name "refname" when referring to
a string that names a reference.
Signed-off-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>