Ensure external users are not able to clone disabled repositories.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/23788

See merge request !2017
Signed-off-by: NRémy Coutable <remy@rymai.me>

Use a permissions of user to access all dependent projects from CI jobs (this also includes a container images, and in future LFS files)

DRY code + fix rubocop

Add more test cases

Append to changelog

DRY changes list

find_url service for merge_requests

use GET for getting merge request links

remove files

rename to get_url_service

reduce loop

add test case for cross project

refactor tiny thing

update changelog

Before this change we always let users push Git data over HTTP before
deciding whether to accept to push. This was different from pushing
over SSH where we terminate a 'git push' early if we already know the
user is not allowed to push.

This change let Git over HTTP follow the same behavior as Git over
SSH. We also distinguish between HTTP 404 and 403 responses when
denying Git requests, depending on whether the user is allowed to know
the project exists.

This reverts commit 530f5158.

See !4892.
Signed-off-by: NRémy Coutable <remy@rymai.me>

This reverts commit 9ca633eb, reversing
changes made to fb229bbf.

- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4892#note_12892160
- This is more consistent.

1. Don't use case statements for dispatch anymore. This leads to a lot
   of duplication, and makes the logic harder to follow.

2. Remove duplicated logic.

    - For example, the `can_push_to_branch?` exists, but we also have a
      different way of checking the same condition within `change_access_check`.

    - This kind of duplication is removed, and the `can_push_to_branch?`
      method is used in both places.

3. Move checks returning true/false to `UserAccess`.

    - All public methods in `GitAccess` now return an instance of
      `GitAccessStatus`. Previously, some methods would return
      true/false as well, which was confusing.

    - It makes sense for these kinds of checks to be at the level of a
      user, so the `UserAccess` class was repurposed for this. The prior
      `UserAccess.allowed?` classmethod is converted into an instance
      method.

    - All external uses of these checks have been migrated to use the
      `UserAccess` class

4. Move the "change_access_check" into a separate class.

    - Create the `GitAccess::ChangeAccessCheck` class to run these
      checks, which are quite substantial.

    - `ChangeAccessCheck` returns an instance of `GitAccessStatus` as
      well.

5. Break out the boolean logic in `ChangeAccessCheck` into `if/else`
   chains - this seems more readable.

6. I can understand that this might look like overkill for !4892, but I
   think this is a good opportunity to clean it up.

    - http://martinfowler.com/bliki/OpportunisticRefactoring.html

1. When a merge request is being merged, save the merge commit SHA in
   the `in_progress_merge_commit_sha` database column.

2. The `pre-receive` hook looks for any locked (in progress) merge
   request with `in_progress_merge_commit_sha` matching the `newrev` it
   is passed.

3. If it finds a matching MR, the merge is legitimate.

4. Update `git_access_spec` to test the behaviour we added here. Also
   refactored this spec a bit to make it easier to add more contexts / conditions.

- Cherry-picked from `mvestergaard:branch-protection-dev-merge`
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4220

Rename ENV['PROTOCOL'] to ENV['GL_PROTOCOL'] to conform to what GitLab Shell expects and make the `protocol` param in `GitAccess` mandatory.

If you attempt to push thousands of branches at once, the 60-second timeout
will occur because GitAccess checking does a lot of work to check if the
user has permission to push to a branch. This changes does two things:

1. Instead of making 1 DB query per branch push, use a memoized list of protected branches to check
2. Memoize what permissions the user has to perform on this project

On a test of 10,000 branch pushes, this prevents gitlab-shell from hitting the 60-second
timeout.

Closes #17225

Previously this method would directly receive the output of tag_name().
This method could either return a String or nil. In the previous setup
this would somehow magically work but because Rugged::TagCollection#[]
doesn't accept nil values it started to fail.

To work around this the elsif in change_access_check() assigns the
result of tag_name() to a local and then _only_ calls protected_tag?()
if the tag name is not nil. The extra parenthesis are put in place to
ensure that things are parsed correctly, without these the code would be
parsed as follows:

    elsif tag_ref = (tag_name(ref) && protected_tag(tag_ref))

During runtime this would basically resolve to:

    elsif tag_ref = (tag_name(ref) && protected_tag(nil))

This is because when you refer to the variable you're assigning _in_ the
assignment Ruby returns nil instead of raising an error.

This removes the need for retrieving the entire list of tags just to
check if a specific one exists.