Closes gitlab-org/gitlab-ce#22330

Post-merge improve of CI permissions

Improves code from !6409

See merge request !6432

Upgrade Sidekiq to 4.2.1, remove dependency on Sinatra

This updates Sidekiq to 4.2.1, which adds full support for Rails 5 by removing a dependency on Sinatra which was one of the remaining Rails 5 blockers.

Major things to check: Sidekiq still works, obviously. Also that the Web UI/Admin dashboard works and doesn't lose any functionality (based on my testing it works fine).

Working toward #14286.

Changelog: https://github.com/mperham/sidekiq/blob/921e939f995fbb5238975d4121d728b95be99ab5/Changes.md#421

See merge request !6349

Cycle Analytics: first iteration

## What does this MR do?

- Implement the first iteration of the "Cycle Analytics" feature.

## What are the relevant issue numbers?

- Closes #21170 

## Screenshots

![cycle_analytics_screencast.gif](/uploads/d23c3c912caa6935fd47b53ca3a56b97/cycle_analytics.gif)

## Backend Tasks

- [x]  Implementation
    - [x]  Phases
        - [x]  Issue (Tracker)
        - [x]  Plan (Board)
        - [x]  Code (IDE)
        - [x]  Test (CI)
        - [x]  Review (MR)
        - [x]  Staging (CD)
        - [x]  Production (Total)
    - [x]  Make heuristics more modular
    - [x]  Scope to project
    - [x]  Date range (30 days, 90 days)
    - [x]  Access restriction
- [x]  Test
    - [x]  Find a better way to test these phases
    - [x]  Phases
        - [x]  Issue (Tracker)
        - [x]  Plan (Board)
        - [x]  Code (IDE)
        - [x]  Test (CI)
        - [x]  Review (MR)
        - [x]  Staging (CD)
        - [x]  Production (Total)
    - [x]  Test for "end case happens before start case"
    - [x]  Consolidate helper
- [x]  Miniboss review
- [x]  Performance testing with mock data
- [x]  Improve performance
    - [x]  Pre-calculate "merge requests closing issues
    - [x]  Pre-calculate everything else
- [x]  Test performance against 10k issues
- [x]  Test all pre-calculation code
    - [x]  Ci::Pipeline -> build start/finish
    - [x]  Ci::Pipeline#merge_requests
    - [x]  Issue -> record default metrics after save
    - [x]  MergeRequest -> record default metrics after save
    - [x]  Deployment -> Update "first_deployed_to_production_at" for MR metrics
    - [x]  Git Push -> Update "first commit mention" for issue metrics
    - [x]  Merge request create/update/refresh -> Update "merge requests closing issues"
- [x]  Remove `MergeRequestsClosingIssues` when necessary
- [x]  Changes to unblock Fatih
    - [x]  Add summary data
    - [x]  `stats` should be array
    - [x]  Let `stats` be `null` if all `stats` are null
- [x]  Indexes for "merge requests closing issues"
- [x]  Test summary data
- [x]  Scope everything to project
    - [x]  Find out why tests were passing
- [x]  Filter should include issues/MRs which have made it to production within the range
- [x]  Don't create duplicate `MergeRequestsClosingIssues`
- [x]  Fix tests
- [x]  MySQL median
- [x]  Assign to Douwe for review
- [x]  Fix conflicts
- [x]  Implement suggestions from Yorick's review
    - [x]  Test on PG
    - [x]  Test on MySQL
- [x]  Refactor
    - [x]  Cleanup
        - [x]  What happens if we have no data at all?
        - [x]  Extract common queries to methods / scopes
    - [x]  Remove unused queries
    - [x]  Downtime for foreign key migrations
    - [x]  Find a way around "if issue.metrics.present?" all over the place
    - [x]  Find a way around "if merge_request.metrics.present?" all over the place
    - [x]  Test migrations on a fresh database
        - [x]  MySQL
        - [x]  Pg
- [x]  Access issues
    - While the project is public and the visibility is set to "Everyone with access", you cannot visit the cycle analytics page when signed out.
- [x]  CHANGELOG
- [x]  Implement suggestions from Douwe's review
    - [x]  First set of comments
    - [x]  Second set of comments
    - [x]  Third set of comments
    - [x]  Fourth set of comments
- [x]  Make sure build is green
- [ ]  Make issue for "polish"
- [ ]  EE MR


See merge request !5986

- Pluralize summary titles
- Remove the `run_query` method - always return sql strings from the
  `date_time_sql` methods

1. The spec tests that if:

  - The merge request is merged
  - The target branch is deployed to production
  - The `first_deployed_to_production_at` metric is `nil` (for some reason)
  - The target branch is deployed to production again
  - The `first_deployed_to_production_at` metric stays as `nil` (and is
    not overwritten).

2. Failure only on MySQL due to some datetime weirdness.

- Instead of overriding `create` and `update` in `MergeRequests::BaseService`
- Get all merge request service specs passing

Test if issue authors can access private projects

See merge request !6419

- Don't use `TableReferences` - using `.arel_table` is shorter!
- Move some database-related code to `Gitlab::Database`
- Remove the `MergeRequest#issues_closed` and
  `Issue#closed_by_merge_requests`  associations. They were either
  shadowing or were too similar to existing methods. They are not being
  used anywhere, so it's better to remove them to reduce confusion.
- Use Rails 3-style validations
- Index for `MergeRequest::Metrics#first_deployed_to_production_at`
- Only include `CycleAnalyticsHelpers::TestGeneration` for specs that
  need it.
- Other minor refactorings.

Add docs on Cycle Analytics

Document Cycle Analytics first iteration https://gitlab.com/gitlab-org/gitlab-ce/issues/21170

See merge request !6437

Fixed slash commands not loading

## What does this MR do?

Fixes an issue with slash commands not working when the autocomplete source is loading & then the new issue button is clicked. This also fixes an issue where the autocomplete source is loaded on pages where it isn't actually needed.

## What are the relevant issue numbers?

Closes #21774, #21807

See merge request !6207

[ci skip]

Fix issues with wrong issues/merge request counts when filters are selected

Closes #15356 plus counter for issues and MR are now displayed for the these paths `https://gitlab.com/groups/group-name/issues`  `https://gitlab.com/groups/group-name/merge_requests`  `https://gitlab.com/dashboard/issues` and  `https://gitlab.com/dashboard/merge_requests`

See merge request !4960

See gitlab-org/gitlab-ce!6247.

Add optional 'author' param when making commits

This fulfills https://gitlab.com/gitlab-org/gitlab-ce/issues/20789 by adding the ability to specify a user's ID to be used as the author of a commit.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/20789

See merge request !5822

Fix note form hint showing slash commands supported for commits

## What are the relevant issue numbers?

Closes #22218

See merge request !6368

Fix processing of events when build finished.

Update pipeline after processing builds. Otherwise we can get into scenario where pipeline will be marked as running.

This solves a quite significant regression in Pipeline processing.

Proper fix is to move all this to Sidekiq Worker and process pipeline there.
I'll do it after 8.12 release.

See merge request !6410

Helper methods are meant for views

Render stages in valid order

Currently stages in stages dropdown on builds page are not in valid order. It is important to display stages in other those are defined in the pipeline.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/21133

See merge request !6430

And `scss_lint`

- Move things common to `Issue` and `MergeRequest` into `Issuable`
- Move more database-specific functions into `Gitlab::Database`
- Indentation changes and other minor refactorings.