Closes gitaly#851

Closes gitaly#863

Closes gitaly#866

Due to the refactoring in !16082, `Blob#batch` no longer falls back
to a default `blob_size_limit`. Since `Repository#batch_blobs` was using
a default `nil` value, this would cause issues in the `Blob#find_by_rugged`
method.

This fix here is to be consistent and use a non-nil default value in
`Repository#batch_blobs`.

The problem was masked in development and tests because Gitaly is always
enabled by default for all features.

Closes #41735

Signed-off-by: NRémy Coutable <remy@rymai.me>

This reverts merge request !15712

By importing this Ruby code into gitlab-rails (and gitaly-ruby), we avoid
200ms of startup time for each gitlab_projects subprocess we are eliminating.

By not having a gitlab_projects subprocess between gitlab-rails / sidekiq and
any git subprocesses (e.g. for fork_project, fetch_remote, etc, calls), we can
also manage these git processes more cleanly, and avoid sending SIGKILL to them

Closes gitaly#808

This also allows us to simplify the naming since we can make some
fetching methods private.

Closes gitaly#780

Closes gitaly#737

This was failing regularly with an Error 500 when the API branches endpoint
was used.

Closes #40615

If a merge request was created with a branch name that also matched a tag name,
we'd generate a comparison to or from the tag respectively, rather than the
branch. Merging would still use the branch, of course.

To avoid this, ensure that when we get the branch heads, we prepend the
reference prefix for branches, which will ensure that we generate the correct
comparison.

After installing a new gem, batch-loader, a construct can be used to
queue data to be fetched in bulk. The gem was also introduced in both
gitlab-org/gitlab-ce!14680 and gitlab-org/gitlab-ce!14846, but those mrs
are not merged yet.

For the generation of diffs, both the old blob and the new blob need to
be loaded. This for every file in the diff, too. Now we collect all
these so we do 1 fetch. Three `.allow_n_plus_1_calls` have been removed,
which I expect to be valid, but this needs to be confirmed by a full CI
run.

Possibly closes:
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37445
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37599
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37431