Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Suppose a fetch or push is requested between two shallow repositories
(with no history deepening or shortening). A pack that contains
necessary objects is transferred over together with .git/shallow of
the sender. The receiver has to determine whether it needs to update
.git/shallow if new refs needs new shallow comits.

The rule here is avoid updating .git/shallow by default. But we don't
want to waste the received pack. If the pack contains two refs, one
needs new shallow commits installed in .git/shallow and one does not,
we keep the latter and reject/warn about the former.

Even if .git/shallow update is allowed, we only add shallow commits
strictly necessary for the former ref (remember the sender can send
more shallow commits than necessary) and pay attention not to
accidentally cut the receiver history short (no history shortening is
asked for)

So the steps to figure out what ref need what new shallow commits are:

1. Split the sender shallow commit list into "ours" and "theirs" list
   by has_sha1_file. Those that exist in current repo in "ours", the
   remaining in "theirs".

2. Check the receiver .git/shallow, remove from "ours" the ones that
   also exist in .git/shallow.

3. Fetch the new pack. Either install or unpack it.

4. Do has_sha1_file on "theirs" list again. Drop the ones that fail
   has_sha1_file. Obviously the new pack does not need them.

5. If the pack is kept, remove from "ours" the ones that do not exist
   in the new pack.

6. Walk the new refs to answer the question "what shallow commits,
   both ours and theirs, are required in .git/shallow in order to add
   this ref?". Shallow commits not associated to any refs are removed
   from their respective list.

7. (*) Check reachability (from the current refs) of all remaining
   commits in "ours". Those reachable are removed. We do not want to
   cut any part of our (reachable) history. We only check up
   commits. True reachability test is done by
   check_everything_connected() at the end as usual.

8. Combine the final "ours" and "theirs" and add them all to
   .git/shallow. Install new refs. The case where some hook rejects
   some refs on a push is explained in more detail in the push
   patches.

Of these steps, #6 and #7 are expensive. Both require walking through
some commits, or in the worst case all commits. And we rather avoid
them in at least common case, where the transferred pack does not
contain any shallow commits that the sender advertises. Let's look at
each scenario:

1) the sender has longer history than the receiver

   All shallow commits from the sender will be put into "theirs" list
   at step 1 because none of them exists in current repo. In the
   common case, "theirs" becomes empty at step 4 and exit early.

2) the sender has shorter history than the receiver

   All shallow commits from the sender are likely in "ours" list at
   step 1. In the common case, if the new pack is kept, we could empty
   "ours" and exit early at step 5.

   If the pack is not kept, we hit the expensive step 6 then exit
   after "ours" is emptied. There'll be only a handful of objects to
   walk in fast-forward case. If it's forced update, we may need to
   walk to the bottom.

3) the sender has same .git/shallow as the receiver

   This is similar to case 2 except that "ours" should be emptied at
   step 2 and exit early.

A fetch after "clone --depth=X" is case 1. A fetch after "clone" (from
a shallow repo) is case 3. Luckily they're cheap for the common case.

A push from "clone --depth=X" falls into case 2, which is expensive.
Some more work may be done at the sender/client side to avoid more
work on the server side: if the transferred pack does not contain any
shallow commits, send-pack should not send any shallow commits to the
receive-pack, effectively turning it into a normal push and avoid all
steps.

This patch implements all steps except #3, already handled by
fetch-pack and receive-pack, #6 and #7, which has their own patch due
to their size.

(*) in previous versions step 7 was put before step 3. I reorder it so
    that the common case that keeps the pack does not need to walk
    commits at all. In future if we implement faster commit
    reachability check (maybe with the help of pack bitmaps or commit
    cache), step 7 could become cheap and be moved up before 6 again.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

No callers pass a non-empty pointer as shallow_points at this
stage. As a result, all clients still refuse to talk to shallow
repository on the other end.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

If either receive-pack or upload-pack is called on a shallow
repository, shallow commits (*) will be sent after the ref
advertisement (but before the packet flush), so that the receiver has
the full "shape" of the sender's commit graph. This will be needed for
the receiver to update its .git/shallow if necessary.

This breaks the protocol for all clients trying to push to a shallow
repo, or fetch from one. Which is basically the same end result as
today's "is_repository_shallow() && die()" in receive-pack and
upload-pack. New clients will be made aware of shallow upstream and
can make use of this information.

The sender must send all shallow commits that are sent in the
following pack. It may send more shallow commits than necessary.

upload-pack for example may choose to advertise no shallow commits if
it knows in advance that the pack it's going to send contains no
shallow commits. But upload-pack is the server, so we choose the
cheaper way, send full .git/shallow and let the client deal with it.

Smart HTTP is not affected by this patch. Shallow support on
smart-http comes later separately.

(*) A shallow commit is a commit that terminates the revision
    walker. It is usually put in .git/shallow in order to keep the
    revision walker from going out of bound because there is no
    guarantee that objects behind this commit is available.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

If we borrow objects from another repository, we should also pay
attention to their $GIT_DIR/shallow (and even info/grafts). But
current alternates code does not.

Reject alternate repos that are shallow because we do not do it
right. In future the alternate code may be updated to check
$GIT_DIR/shallow properly so that this restriction could be lifted.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

send-pack can send a pack with loose ends to the server.  receive-pack
before 6d4bb383 (fetch: verify we have everything we need before
updating our ref - 2011-09-01) does not detect this and keeps the pack
anyway, which corrupts the repository, at least from fsck point of
view.

send-pack will learn to safely push from a shallow repository later.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The latter can do everything the former can and is used in many more
places.
Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NNguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

* maint:
  remote-hg: don't decode UTF-8 paths into Unicode objects

The internal mercurial API expects ordinary 8-bit string objects, not
Unicode string objects.  With this change, the test-hg.sh unit tests
pass again.
Signed-off-by: NRichard Hansen <rhansen@bbn.com>
Reviewed-by: NFelipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Unbreaks a recent breakage due to use of unquote-c-style.

This may need to be cherry-picked down to 1.8.4.x series.

* 'rh/remote-hg-bzr-updates' (early part):
  remote-hg: don't decode UTF-8 paths into Unicode objects

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

* mb/relnotes-1.8.5-fix:
  RelNotes: spelling & grammar fixes

Signed-off-by: NMarc Branchaud <marcnarc@xiplink.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Fixes a regression on 'master' since v1.8.4.

* nd/literal-pathspecs:
  pathspec: stop --*-pathspecs impact on internal parse_pathspec() uses

The internal mercurial API expects ordinary 8-bit string objects, not
Unicode string objects.  With this change, the test-hg.sh unit tests
pass again.
Signed-off-by: NRichard Hansen <rhansen@bbn.com>
Reviewed-by: NFelipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Hotfix for recent regression while talking to upload-pack
in a repository with many symbolic refs.

* maint:
  Revert "upload-pack: send non-HEAD symbolic refs"

Hot-fix for a regression.

* jx/branch-vv-always-compare-with-upstream:
  branch: fix --verbose output column alignment

Commit f2e08739 (branch: report invalid tracking branch as gone) removed
an early return from fill_tracking_info() in the path taken when 'git
branch -v' lists a branch in sync with its upstream. This resulted in an
unconditionally added space in front of the subject line:

    $ git branch -v
    * master f5eb3da  commit pushed to upstream
      topic  f935eb6 unpublished topic

Instead, only add the trailing space if a decoration have been added.

To catch this kind of whitespace breakage in the tests, be a bit less
smart when filtering the output through sed.
Signed-off-by: NTorstein Hegge <hegge@resisty.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This reverts commit 5e7dcad7; there
may be unbounded number of symbolic refs in the repository, but the
capability header line in the on-wire protocol has a rather low
length limit.

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

* git://github.com/git-l10n/git-po:
  l10n: de.po: improve error message when pushing to unknown upstream
  l10n: de.po: translate 68 new messages
  po/TEAMS: update Thomas Rast's email address
  l10n: Update Swedish translation (2194t0f0u)
  l10n: fr.po 2194/1294 messages translated
  l10n: zh_CN.po: translate 68 messages (2194t0f0u)
  l10n: vi.po (2194t): Update and minor fix
  l10n: git.pot: v1.8.5 round 1 (68 new, 9 removed)

"timezone" is two words, not one (i.e. "time zone" is correct).

Correct this in these files:
-- date-formats.txt
-- git-blame.txt
-- git-cvsimport.txt
-- git-fast-import.txt
-- git-svn.txt
-- gitweb.conf.txt
-- rev-list-options.txt
Signed-off-by: NJason St. John <jstjohn@purdue.edu>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Signed-off-by: NRalf Thielow <ralf.thielow@gmail.com>
Acked-by: NThomas Rast <tr@thomasrast.ch>

Translate 68 new messages came from git.pot update in 727b9576
(l10n: git.pot: v1.8.5 round 1 (68 new, 9 removed)).
Signed-off-by: NRalf Thielow <ralf.thielow@gmail.com>
Acked-by: NThomas Rast <tr@thomasrast.ch>

Signed-off-by: NRalf Thielow <ralf.thielow@gmail.com>
Acked-by: NThomas Rast <tr@thomasrast.ch>

* sv/nafmo/master:
  l10n: Update Swedish translation (2194t0f0u)

And fix a typo.
Signed-off-by: NPeter Krefting <peter@softwolves.pp.se>

Signed-off-by: NJean-Noel Avila <jn.avila@free.fr>
Signed-off-by: NSebastien Helleu <flashcode@flashtux.org>

Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The interaction between use of Perl in our test suite and NO_PERL
has been clarified a bit.

* jn/test-prereq-perl-doc:
  t/README: tests can use perl even with NO_PERL

A fast-import stream expresses a pathname with funny characters by
quoting them in C style; remote-hg remote helper (in contrib/)
forgot to unquote such a path.

* ap/remote-hg-unquote-cquote:
  remote-hg: unquote C-style paths when exporting

One long-standing flaw in the pack transfer protocol used by "git
clone" was that there was no way to tell the other end which branch
"HEAD" points at, and the receiving end needed to guess.  A new
capability has been defined in the pack protocol to convey this
information so that cloning from a repository with more than one
branches pointing at the same commit where the HEAD is at now
reliably sets the initial branch in the resulting repository.

* jc/upload-pack-send-symref:
  t5570: Update for clone-progress-to-stderr branch
  t5570: Update for symref capability
  clone: test the new HEAD detection logic
  connect: annotate refs with their symref information in get_remote_head()
  connect.c: make parse_feature_value() static
  upload-pack: send non-HEAD symbolic refs
  upload-pack: send symbolic ref information as capability
  upload-pack.c: do not pass confusing cb_data to mark_our_ref()
  t5505: fix "set-head --auto with ambiguous HEAD" test

We did not handle cases where http transport gets redirected during
the authorization request (e.g. from http:// to https://).

* jk/http-auth-redirects:
  http.c: Spell the null pointer as NULL
  remote-curl: rewrite base url from info/refs redirects
  remote-curl: store url as a strbuf
  remote-curl: make refs_url a strbuf
  http: update base URLs when we see redirects
  http: provide effective url to callers
  http: hoist credential request out of handle_curl_result
  http: refactor options to http_get_*
  http_request: factor out curlinfo_strbuf
  http_get_file: style fixes

* maint:
  Start preparing for 1.8.4.3
  gitignore.txt: fix documentation of "**" patterns