Change prune_ref to delete the ref using a ref transaction. To do this we also
need to add a new flag REF_ISPRUNING that will tell the transaction that we
do not want to delete this ref from the packed refs. This flag is private to
refs.c and not exposed to external callers.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

lock_ref_sha1 was only called from one place in refs.c and only provided
a check that the refname was sane before adding back the initial "refs/"
part of the ref path name, the initial "refs/" that this caller had already
stripped off before calling lock_ref_sha1.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since we only call update_ref_write from a single place and we only call it
with onerr==QUIET_ON_ERR we can just as well get rid of it and just call
write_ref_sha1 directly. This changes the return status for _commit from
1 to -1 on failures when writing to the ref. Eventually we will want
_commit to start returning more detailed error conditions than the current
simple success/failure.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Since we now only call update_ref_lock with onerr==QUIET_ON_ERR we no longer
need this function and can replace it with just calling lock_any_ref_for_update
directly.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

No external callers reference lock_ref_sha1 any more so let's declare it
static.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Switch to using ref transactions in walker_fetch(). As part of the refactoring
to use ref transactions we also fix a potential memory leak where in the
original code if write_ref_sha1() would fail we would end up returning from
the function without free()ing the msg string.

Note that this function is only called when fetching from a remote HTTP
repository onto the local (most of the time single-user) repository which
likely means that the type of collisions that the previous locking would
protect against and cause the fetch to fail for are even more rare.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Wrap all the ref updates inside a transaction.

In the new API there is no distinction between failure to lock and
failure to write a ref.  Both can be permanent (e.g., a ref
"refs/heads/topic" is blocking creation of the lock file
"refs/heads/topic/1.lock") or transient (e.g., file system full) and
there's no clear difference in how the client should respond, so
replace the two statuses "failed to lock" and "failed to write" with
a single status "failed to update ref".  In both cases a more
detailed message is sent by sideband to diagnose the problem.

Example, before:

 error: there are still refs under 'refs/heads/topic'
 remote: error: failed to lock refs/heads/topic
 To foo
  ! [remote rejected] HEAD -> topic (failed to lock)

After:

 error: there are still refs under 'refs/heads/topic'
 remote: error: Cannot lock the ref 'refs/heads/topic'.
 To foo
  ! [remote rejected] HEAD -> topic (failed to update ref)
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change the update_ref helper function to use a ref transaction internally.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change create_branch to use a ref transaction when creating the new branch.

This also fixes a race condition in the old code where two concurrent
create_branch could race since the lock_any_ref_for_update/write_ref_sha1
did not protect against the ref already existing. I.e. one thread could end up
overwriting a branch even if the forcing flag is false.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change update_branch() to use ref transactions for updates.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change to use ref transactions for all updates to refs.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change commit.c to use ref transactions for all ref updates.
Make sure we pass a NULL pointer to ref_transaction_update if have_old
is false.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Update replace.c to use ref transactions for updates.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change tag.c to use ref transactions for all ref updates.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Track the state of a transaction in a new state field. Check the field for
sanity, i.e. that state must be OPEN when _commit/_create/_delete or
_update is called or else die(BUG:...)
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Add an err argument to _begin so that on non-fatal failures in future ref
backends we can report a nice error back to the caller.
While _begin can currently never fail for other reasons than OOM, in which
case we die() anyway, we may add other types of backends in the future.
For example, a hypothetical MySQL backend could fail in _begin with
"Can not connect to MySQL server. No route to host".
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Change ref_transaction_delete() to do basic error checking and return
non-zero on error. Update all callers to check the return for
ref_transaction_delete(). There are currently no conditions in _delete that
will return error but there will be in the future. Add an err argument that
will be updated on failure.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Do basic error checking in ref_transaction_create() and make it return
non-zero on error. Update all callers to check the result of
ref_transaction_create(). There are currently no conditions in _create that
will return error but there will be in the future. Add an err argument that
will be updated on failure.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Reviewed-by: NMichael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Update ref_transaction_update() do some basic error checking and return
non-zero on error. Update all callers to check ref_transaction_update() for
error. There are currently no conditions in _update that will return error but
there will be in the future. Add an err argument that will be updated on
failure. In future patches we will start doing both locking and checking
for name conflicts in _update instead of _commit at which time this function
will start returning errors for these conditions.

Also check for BUGs during update and die(BUG:...) if we are calling
_update with have_old but the old_sha1 pointer is NULL.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Since all callers now use QUIET_ON_ERR we no longer need to provide an onerr
argument any more. Remove the onerr argument from the ref_transaction_commit
signature.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Call ref_transaction_commit with QUIET_ON_ERR and use the strbuf that is
returned to print a log message if/after the transaction fails.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Change update_ref_write to also update an error strbuf on failure.
This makes the error available to ref_transaction_commit callers if the
transaction failed due to update_ref_sha1/write_ref_sha1 failures.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Make ref_update_reject_duplicates return any error that occurs through a
new strbuf argument. This means that when a transaction commit fails in
this function we will now be able to pass a helpful error message back to the
caller.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno from write_ref_sha1() meaningful, which should fix

* a bug in "git checkout -b" where it prints strerror(errno)
  despite errno possibly being zero or clobbered

* a bug in "git fetch"'s s_update_ref, which trusts the result of an
  errno == ENOTDIR check to detect D/F conflicts
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from resolve_ref_unsafe() meaningful,
which should fix

 * a bug in lock_ref_sha1_basic, where it assumes EISDIR
   means it failed due to a directory being in the way
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from commit_packed_refs() meaningful,
which should fix

 * a bug in "git clone" where it prints strerror(errno) based on
   errno, despite errno possibly being zero and potentially having
   been clobbered by that point
 * the same kind of bug in "git pack-refs"

and prepares for repack_without_refs() to get a meaningful
error message when commit_packed_refs() fails without falling into
the same bug.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from remove_empty_directories() more
obviously meaningful, which should provide some peace of mind for
people auditing lock_ref_sha1_basic.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from verify_lock() meaningful, which
should almost but not completely fix

 * a bug in "git fetch"'s s_update_ref, which trusts the result of an
   errno == ENOTDIR check to detect D/F conflicts

ENOTDIR makes sense as a sign that a file was in the way of a
directory we wanted to create.  Should "git fetch" also look for
ENOTEMPTY or EEXIST to catch cases where a directory was in the way
of a file to be created?
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from log_ref_setup() meaningful,
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Update repack_without_refs to take an err argument and update it if there
is a failure. Pass the err variable from ref_transaction_commit to this
function so that callers can print a meaningful error message if _commit
fails due to this function.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Making errno when returning from lock_file() meaningful, which should
fix

 * an existing almost-bug in lock_ref_sha1_basic where it assumes
   errno==ENOENT is meaningful and could waste some work on retries

 * an existing bug in repack_without_refs where it prints
   strerror(errno) and picks advice based on errno, despite errno
   potentially being zero and potentially having been clobbered by
   that point
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Introducing a new unable_to_lock_message helper, which has nicer
semantics than unable_to_lock_error and cleans up lockfile.c a little.
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Add a strbuf argument to _commit so that we can pass an error string back to
the caller. So that we can do error logging from the caller instead of from
_commit.

Longer term plan is to first convert all callers to use onerr==QUIET_ON_ERR
and craft any log messages from the callers themselves and finally remove the
onerr argument completely.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Allow ref_transaction_free(NULL) as a no-op. This makes ref_transaction_free
easier to use and more similar to plain 'free'.

In particular, it lets us rollback unconditionally as part of cleanup code
after setting 'transaction = NULL' if a transaction has been committed or
rolled back already.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

ref_transaction_create|delete|update has no need to modify the sha1
arguments passed to it so it should use const unsigned char* instead
of unsigned char*.

Some functions, such as fast_forward_to(), already have its old/new
sha1 arguments as consts. This function will at some point need to
use ref_transaction_update() in which case this change is required.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

We do not yet need both a rollback and a free function for transactions.
Remove ref_transaction_rollback and use ref_transaction_free instead.

At a later stage we may reintroduce a rollback function if we want to start
adding reusable transactions and similar.
Reviewed-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NRonnie Sahlberg <sahlberg@google.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>
Acked-by: NMichael Haggerty <mhagger@alum.mit.edu>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Allow specifying only certain individual test pieces to be run
using a range notation (e.g. "t1234-test.sh --run='1-4 6 8 9-'").

* ib/test-selectively-run:
  t0000-*.sh: fix the GIT_SKIP_TESTS sub-tests
  test-lib: '--run' to run only specific tests
  test-lib: tests skipped by GIT_SKIP_TESTS say so
  test-lib: document short options in t/README