When a project is forked, the new repository used to be a deep copy of everything
stored on disk by leveraging `git clone`. This works well, and makes isolation
between repository easy. However, the clone is at the start 100% the same as the
origin repository. And in the case of the objects in the object directory, this
is almost always going to be a lot of duplication.

Object Pools are a way to create a third repository that essentially only exists
for its 'objects' subdirectory. This third repository's object directory will be
set as alternate location for objects. This means that in the case an object is
missing in the local repository, git will look in another location. This other
location is the object pool repository.

When Git performs garbage collection, it's smart enough to check the
alternate location. When objects are duplicated, it will allow git to
throw one copy away. This copy is on the local repository, where to pool
remains as is.

These pools have an origin location, which for now will always be a
repository that itself is not a fork. When the root of a fork network is
forked by a user, the fork still clones the full repository. Async, the
pool repository will be created.

Either one of these processes can be done earlier than the other. To
handle this race condition, the Join ObjectPool operation is
idempotent. Given its idempotent, we can schedule it twice, with the
same effect.

To accommodate the holding of state two migrations have been added.
1. Added a state column to the pool_repositories column. This column is
managed by the state machine, allowing for hooks on transitions.
2. pool_repositories now has a source_project_id. This column in
convenient to have for multiple reasons: it has a unique index allowing
the database to handle race conditions when creating a new record. Also,
it's nice to know who the host is. As that's a short link to the fork
networks root.

Object pools are only available for public project, which use hashed
storage and when forking from the root of the fork network. (That is,
the project being forked from itself isn't a fork)

In this commit message I use both ObjectPool and Pool repositories,
which are alike, but different from each other. ObjectPool refers to
whatever is on the disk stored and managed by Gitaly. PoolRepository is
the record in the database.

Use shelling out to git to write refs instead of rugged, hoping to
avoid creating invalid refs.

To update HEAD we switched to using `git symbolic-ref`.

Signed-off-by: NTakuya Noguchi <takninnovationresearch@gmail.com>

This reverts merge request !23229

This reverts merge request !23140

Now only the data was shown of the service, which is not valueable at
times given some of those expose a lot of RPCs.

Enables frozen string for the following:

* lib/gitlab/fogbugz_import/**/*.rb
* lib/gitlab/gfm/**/*.rb
* lib/gitlab/git/**/*.rb
* lib/gitlab/gitaly_client/**/*.rb
* lib/gitlab/gitlab_import/**/*.rb
* lib/gitlab/google_code_import/**/*.rb
* lib/gitlab/gpg/**/*.rb
* lib/gitlab/grape_logging/**/*.rb
* lib/gitlab/graphql/**/*.rb
* lib/gitlab/graphs/**/*.rb
* lib/gitlab/hashed_storage/**/*.rb
* lib/gitlab/health_checks/**/*.rb

Partially address gitlab-org/gitlab-ce#47424.

This new endpoint allow users to update a submodule's reference.

The MR involves adding a new operation RPC operation in gitaly-proto
(see gitlab-org/gitaly-proto!233) and change Gitaly to use this
new version (see gitlab-org/gitaly!936).

See gitlab-org/gitlab-ce!20949

This indirection doesn't provide any value, so remove it

Having this in a concern allows us to reuse it for different single
purpose classes that call out to git without going through the
repository every time.

As we now support getting the merge base for multiple revisions in
gitaly, we can provide this functionality in our API

Was introduced in the time that GitLab still used NFS, which is not
required anymore in most cases. By removing this, the API it calls will
return empty responses. This interface has to be removed in the next
major release, expected to be 12.0.

Cleanup code, and refactor tests that still use Rugged. After this, there should
be no Rugged code that access the instance's repositories on non-test
environments. There is still some rugged code for other tasks like the
repository import task, but since it doesn't access any repository storage path
it can stay.

Implements list_last_commits_for_tree to communicate with the
ListLastCommitsForTree Gitaly RPC

Bumps the Gitaly server version

Bumps the Gitaly-Proto gem version

This adds a basic interface to fetch diff statistics given two SHAs.
It's a requirement for #49399 #20282 and #19232.

This has been tested on .com. So far no errors have been seen.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1310
Closes https://gitlab.com/gitlab-org/gitaly/issues/1246

Introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20087,
this has been tested on .com now, and is stable.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1286
Closes https://gitlab.com/gitlab-org/gitaly/issues/1233

This old migration used Rugged to find a commit, while Gitaly is the
prefered way now. By migrating this to Gitaly, Gitaly is now a required
running component for this migration.

Part of https://gitlab.com/gitlab-org/gitaly/issues/1106

After trying to remove the whole method in
8f69014af2902d8d53fe931268bec60f6858f160, this is a more gentle
approach to the method. :)

Prior to this change, new commit detection wasn't implemented in Gitaly,
this was done through: https://gitlab.com/gitlab-org/gitaly/merge_requests/779

As the new implemented got moved around a bit, the whole RevList class
got removed.

Part of https://gitlab.com/gitlab-org/gitaly/issues/1233