Don't try to highlight and cache files hidden by .gitattributes entries.

Coverage was failing, so this could slip through the cracks

This adds counters for build artifacts and LFS objects, and moves
the preexisting repository_size and commit_count from the projects
table into a new project_statistics table.

The counters are displayed in the administration area for projects
and groups, and also available through the API for admins (on */all)
and normal users (on */owned)

The statistics are updated through ProjectCacheWorker, which can now
do more granular updates with the new :statistics argument.

- Use Route#request_method instead of Route#route_method
- Use Route#path instead of Route#route_path
Signed-off-by: NRémy Coutable <remy@rymai.me>

I mistakenly concluded Rack::Multipart injects File instances into the
params. These should be UploadedFile instances. This reuses a mock
UploadedFile class we already had in GitLab.

Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

The reason is that Gitea plan to be GitHub-compatible so it makes sense
to just modify GitHubImport a bit for now, and hopefully we can change
it to GitHubishImport once Gitea is 100%-compatible.
Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

- `raise "string"` raises a `RuntimeError` - no need to be explicit
- Remove top-level comment in the `RevList` class
- Use `%w()` instead of `%w[]`
- Extract an `environment_variables` method to cache `env.slice(*ALLOWED_VARIABLES)`
- Use `start_with?` for env variable validation instead of regex match
- Validation specs for each allowed environment variable were identical. Build them dynamically.
- Minor change to `popen3` expectation.

- Don't define "allowed environment variables" in two places.
- Dispatch to different arities of `Popen.open` without an if/else block.
- Use `described_class` instead of explicitly stating the class name within a
- spec.
- Remove `git_environment_variables_validator_spec` and keep the validation inline.

Previously, we were calling out to `popen` without asserting on the returned
exit-code. Now we raise a `RuntimeError` if the exit code is non-zero.

The list of environment variables in `Gitlab::Git::RevList` need to be validate
to make sure that they don't reference any other project on disk.

This commit mixes in `ActiveModel::Validations` into `Gitlab::Git::RevList`, and
validates that the environment variables are on the level (using a custom
validator class). If the validations fail, the force push is still executed
without any environment variables set.

Add specs for the validation using shared examples.

Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8088#note_20133176

Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8088#note_20133505

Ruby 2.1 requires a basename argument to `Tempfile.open`, so just call
it something that makes sense in context for the spec.

- The `scopes_form` partial can be used in the `admin/applications` view
  as well

- Don't allow partials to access instance variables directly. Instead, pass
  in the instance variables as local variables, and use `local_assigns.fetch`
  to assert that the variables are passed in as expected.

- Change a few instances of `render :partial` to `render`

- Remove an instance of `required: false` in a view, since this is the default

- Inline many instances of a local variable (`ip = 'ip'`) in `auth_spec`

- This module is used for git-over-http, as well as JWT.

- The only valid scope here is `api`, currently.

should be correct since it's already passing the
validation anyway.

Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8088#note_20076187

Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8088#note_20062470

Eventually we should move to SafeYAML, but requiring that
would impact all other `YAML.load` which is bad. For this
particular case, I think we could just check it strictly.

Fix missing Note access checks in by moving Note#search to updated NoteFinder

Split from !2024 to partially solve https://gitlab.com/gitlab-org/gitlab-ce/issues/23867

## Which fixes are in this MR?

⚠ - Potentially untested  
💣 - No test coverage  
🚥 - Test coverage of some sort exists (a test failed when error raised)  
🚦 - Test coverage of return value (a test failed when nil used)  
✅ - Permissions check tested

### Note lookup without access check

- [x] ✅ app/finders/notes_finder.rb:13 :download_code check
- [x] ✅ app/finders/notes_finder.rb:19 `SnippetsFinder`
- [x] ✅ app/models/note.rb:121 [`Issue#visible_to_user`]
- [x] ✅ lib/gitlab/project_search_results.rb:113
  - This is the only use of `app/models/note.rb:121` above, but importantly has no access checks at all. This means it leaks MR comments and snippets when those features are `team-only` in addition to the issue comments which would be fixed by `app/models/note.rb:121`.
  - It is only called from SearchController where `can?(current_user, :download_code, @project)` is checked, so commit comments are not leaked.

### Previous discussions
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_13_13 `: download_code` check on commit
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_19_19 `SnippetsFinder` should be used
  - `SnippetsFinder` should check if the snippets feature is enabled -> https://gitlab.com/gitlab-org/gitlab-ce/issues/25223

###  Acceptance criteria met?
- [x] Tests added for new code
- [x] TODO comments removed
- [x] Squashed and removed skipped tests
- [x] Changelog entry
- [ ] State Gitlab versions affected and issue severity in description
- [ ] Create technical debt issue for NotesFinder.
  - Either split into `NotesFinder::ForTarget` and `NotesFinder::Search` or consider object per notable type such as `NotesFinder::OnIssue`. For the first option could create `NotesFinder::Base` which is either inherited from or which can be included in the other two.
  - Avoid case statement anti-pattern in this finder with use of `NotesFinder::OnCommit` etc. Consider something on the finder for this? `Model.finder(user, project)`
  - Move `inc_author` to the controller, and implement `related_notes` to replace `non_diff_notes`/`mr_and_commit_notes`

See merge request !2035