GitLab Performance Monitoring is now able to track custom events not
directly related to application performance. These events include the
number of tags pushed, repositories created, builds registered, etc.

The use of these events is to get a better overview of how a GitLab
instance is used and how that may affect performance. For example, a
large number of Git pushes may have a negative impact on the underlying
storage engine.

Events are stored in the "events" measurement and are not prefixed with
"rails_" or "sidekiq_", this makes it easier to query events with the
same name triggered from different parts of the application. All events
being stored in the same measurement also makes it easier to downsample
data.

Currently the following events are tracked:

* Creating repositories
* Removing repositories
* Changing the default branch of a repository
* Pushing a new tag
* Removing an existing tag
* Pushing a commit (along with the branch being pushed to)
* Pushing a new branch
* Removing an existing branch
* Importing a repository (along with the URL we're importing)
* Forking a repository (along with the source/target path)
* CI builds registered (and when no build could be found)
* CI builds being updated
* Rails and Sidekiq exceptions

Fixes gitlab-org/gitlab-ce#13720

Use just SQL to check is a user can admin_issue on a project

Tradeoff

- we duplicate how we check admin_issue in a SQL relation in the Ability class

[ci skip]

This ensures that SVGs greater than 2 megabytes are not scrubbed and
rendered. This in turn prevents requests from timing out due to
reading/scrubbing large SVGs potentially taking a lot of time (and
memory). The use of 2 megabytes is completely arbitrary.

Fixes gitlab-org/gitlab-ce#1435

Closes #20828

Currently when a user performs an update of a file through the UI  and there
has already been a change committed to the file the previous commits will be
overwritten without a check to see if the file has been changed.

This commit uses the last commit sha at the time the user starts editing the
file and compares it with the current sha of the file being edited to ensure
they are the same before committing the file. If the shas do not match we
throw an exception preventing the commit from the commit from occurring.

Fixes #5857

In Banzai::Filter::SanitizationFilter#customize_whitelist, we append
three lambdas that has reference to the SanitizationFilter instance,
which in turn (potentially) has a reference to the following chain:

context hash -> Project instance -> Repository instance -> lookup hash
-> various Rugged instances -> various mmap-ed git pack files.

All of the above is not garbage collected because the array we append
the lambdas to is the constant
HTML::Pipeline::SanitizationFilter::WHITELIST.

Because this method is a Rails scope we have to instrument it manually
as regular the instrumentation methods only instrument methods defined
directly on a Class or Module.

There is a race condition in DestroyGroupService now that projects are deleted asynchronously:

1. User attempts to delete group
2. DestroyGroupService iterates through all projects and schedules a Sidekiq job to delete each Project
3. DestroyGroupService destroys the Group, leaving all its projects without a namespace
4. Projects::DestroyService runs later but the can?(current_user,
   :remove_project) is `false` because the user no longer has permission to
   destroy projects with no namespace.
5. This leaves the project in pending_delete state with no namespace/group.

Projects without a namespace or group also adds another problem: it's not possible to destroy the container
registry tags, since container_registry_path_with_namespace is the wrong value.

The fix is to destroy the group asynchronously and to run execute directly on Projects::DestroyService.

Closes #17893

update CHANGELOG

add auto-completion into pipeline

add auto-completion into pipeline

add auto-completion into pipeline

update changelog

modify tests

remove empty lines

add auto-completion into pipeline

update changelog

modify tests

switch text_field_tag into text_field

add test to new field

switch context into describe

Update CHANGELOG

render only commit title

update CHANGELOG

add auto-completion into pipeline

add auto-completion into pipeline

add auto-completion into pipeline

update changelog

modify tests

remove empty lines

add auto-completion into pipeline

update changelog

modify tests

update changelog

Update CHANGELOG

add indetation

add tests to pipeline ref

change file name for tests

change file name for spec tests

remove empty line

rename test it

rename test name

removing unexpected changes

removing unexpected changes2

update changelog

DRY code + fix rubocop

Add more test cases

Append to changelog

DRY changes list

find_url service for merge_requests

use GET for getting merge request links

remove files

rename to get_url_service

reduce loop

add test case for cross project

refactor tiny thing

update changelog

This change simplifies a Pipeline processing by introducing a special new status: created.
This status is used for all builds that are created for a pipeline.
We are then processing next stages and queueing some of the builds (created -> pending) or skipping them (created -> skipped).
This makes it possible to simplify and solve a few ordering problems with how previously builds were scheduled.
This also allows us to visualise a full pipeline (with created builds).

This also removes an after_touch used for updating a pipeline state parameters.
Right now in various places we explicitly call a reload_status! on pipeline to force it to be updated and saved.

Signed-off-by: NRémy Coutable <remy@rymai.me>

One can see which indexes are used in PostgreSQL by running the
following query:

    SELECT relname as table_name, indexrelname as index_name, idx_scan, idx_tup_read, idx_tup_fetch, pg_size_pretty(pg_relation_size(indexrelname::regclass))
    FROM pg_stat_all_indexes
    WHERE schemaname = 'public'
    AND "idx_scan" = 0
    ORDER BY pg_relation_size(indexrelname::regclass) desc;

Using this query I built a list of indexes that could be potentially
removed. After checking every single one by hand to make sure they
really aren't used I only found 1 index that _would_ be used. This was a
GitLab GEO index (EE) specific that's currently not used simply because
the table is empty.

Apart from this one index all indexes could be removed. The migration
also takes care of 6 composite indexes that can be replaced with a
single column index, which in most cases was already present.

For more information see gitlab-org/gitlab-ce#20767.

Since version ruby-2.2.0, method `File::exists?` is deprecated.

Also, mutualize AccessRequests and Members endpoints for Group &
Project.
New API documentation for the AccessRequests endpoints.
Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

These indexes are only used when you search for runners in the admin
interface. This operation is so rarely used that it does not make sense
to slow down every update in order to update the GIN trigram indexes.

Removing these indexes should speed up queries such as those used for
updating the last contact time of CI runners. Locally the timings of
this query were reduced from ~50 ms to ~25 ms:

    UPDATE ci_runners SET updated_at = now(), contacted_at = now();