Increase sort options for issues list from updated_at and create_at,
to include more options close to what is required in actual issue list
UI.

This helps us to use REST API for issues list with sorting capabilities

https://gitlab.com/gitlab-org/gitlab-ce/issues/57402

Root namespaces have storage statistics.
This commit allows namespace owners to get those stats via GraphQL
queries like the following one

{
  namespace(fullPath: "a_namespace_path") {
    rootStorageStatistics {
      storageSize
      repositorySize
      lfsObjectsSize
      buildArtifactsSize
      packagesSize
      wikiSize
    }
  }
}

This replicates Sidekiq behavior
of pushing dead job into DeadSet.

Ensure the evaluation of right-hand side expression always
results in the returning of an object or an empty String

Transform `CancelledError` into `JobRetry::Skip`

- Retry connection when it fails
- Properly shutdown daemon
- Stop monitor if the Exception is raised
- Properly guard exception handling

If we process message that is not designated to us
previously we would fire a separate Thread for that.
We don't need to do it. We can cheaply check if thread
is available, if it is, we can perform expensive operation
then.

This makes:
- very shallow `Middleware::Monitor` to only request tracking
  of sidekiq jobs,
- `SidekiqStatus::Monitor` to be responsible to maintain persistent
  connection to receive messages,
- `SidekiqStatus::Monitor` to always use structured logging
  and instance variables

This adds a middleware to track all threads
for running jobs.

This makes sidekiq to watch for redis-delivered notifications.

This makes be able to send notification to interrupt
running sidekiq jobs.

This does not take into account any native code,
as `Thread.raise` generates exception once the control gets
back to Ruby.

The separate measure should be taken to interrupt gRPC, shellouts,
or anything else that escapes Ruby.

Adding, destroying and toggling emoji previously lacked services and
instead were performed through methods called on Awardable models.

This led to inconsistencies where relevant todos would be marked as done
only when emoji were awarded through our controllers, but not through
the API. Todos could also be marked as done when an emoji was being
removed.

Behaviour changes

- Awarding emoji through the API will now mark a relevant Todo as done
- Toggling an emoji off (destroying it) through our controllers will no
  longer mark a relevant Todo as done

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63372

- Added Gitlab::Ci::Config::Entry::Rules and Gitlab::Ci::Config::Entry::Rules:Rule
  to handle lists of Rule objects to be evalauted for job inclusion
- Added `if:` and `changes:` as available Rules::Rule::Clause classes
- Added Rules handling logic to Seed::Build#included? with extra specs
- Use DisallowedKeysValidator to mutually exclude rules: from only:/except: on job config

Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses
`remote_ip` and `path` for the same fields. Let's standardize these
namings to make it easier for people working with the data.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167

- Add mail interceptor the signs outgoing email with SMIME
- Add lib and helpers to work with SMIME data
- New configuration params for setting up SMIME key and cert files

Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/66319.

The virtus project has been discontinued:

https://github.com/solnic/virtus/commit/a6f896984

This allows exceptions to advertise their support for sentry
and provide structured data.

After moving the multiproc dir cleanup into `config.ru`:`warmup`, we
stopped cleaning Sidekiq metrics dir which is not correct.
This MR intended to fix that. More details:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31668

This functionality is available in the /autocomplete users pseudo-API.
We're attempting to replace that with the canonical API, so it needs
support for this parameter too.

In GitLab 9.0,
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9661 removed the
`subscribed` flag from the API when the user requested a list of issues
or merge requests since calculating this value triggers extensive
Markdown processing.

In GitLab 12.0 via a4fbf39e, we accidentally reintroduced this
performance regression by changing `IssueBasic` to `Issue` in
`entities.rb`. This showed up as a Gitaly N+1 issue since the Markdown
processing would attempt to extract a commit if it detected a regex that
matched a commit.

We restore the prior behavior by once again removing the `subscribed`
flag for the bulk list of issues and merge requests and add a test to
ensure they aren't reintroduced.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66202

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31741 introduced
a regression where not all the right parameters would be passed into
`Ci::CreatePipelineService`. We fix this by breaking out the pipeline
parameters and reusing a method from `Gitlab::DataBuilder::Push`.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66196

Previously `ProjectCacheWorker` would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. `ProjectCacheWorker` would expire
three items:

1. Repository size: This only needs to be updated once per push.
2. Commit count: This only needs to be updated if the default branch
   is updated.
3. Project method caches: This only needs to be updated if the default
   branch changes, but only if certain files change (e.g. README,
   CHANGELOG, etc.).

Because the third item requires looking at the actual changes in the
commit deltas, we schedule one `ProjectCacheWorker` to handle the first
two cases, and schedule a separate `ProjectCacheWorker` for the third
case if it is needed. As a result, this brings down the number of
`ProjectCacheWorker` jobs from N to 2.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046

This removes the group_overview_security_dashboard feature flag

This extends our existing `Gitlab::UsageDataCounters::NoteCounter` to
also count notes on commits and merge requests

This allows us to check inclusion for the *_exists? methods without
downloading the full list of branch names, which is over 100KiB in size
for gitlab-ce at the moment.

- move uploads created by AttachmentUploader
- handle also files created for legacy_diff_notes

https://gitlab.com/gitlab-org/gitlab-ce/issues/62971

Adds support for embedding specific charts from the
metrics dashboard. Expected parameters are dashboard,
title, group, and y_label.

As per https://gitlab.com/gitlab-org/gitlab-ce/issues/46043, project
templates should be squashed before updating, so that repositories
created from these templates don't include the full history of the
backing repository.

This will avoid Danger to suggest a database review for files inside
db/fixtures

- Adds UI to configure in group and project settings
- Removes notification configuration for users when
disabled at group or project level

Previously, the API to retrieve discussions from merge requests often
generated hundreds of Gitaly calls to determine whether a system note
should be shown to the user. It did this by:

1. Rendering the Markdown
2. Extracting cross-references from the Markdown
3. For cross-references that were commits, a Gitaly FindCommit RPC
   would be issued to validate that the commit exists.

The last step is unnecessary because we don't need to display a commit
if the user doesn't have access to the project in the first place.

`RendersNotes#prepare_notes_for_rendering` is already used in
`MergeRequestsController`, which is why we don't see N+1 Gitaly calls
there. We use it here to optimize the note redaction process.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65957

This was added in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31789

CE-specific changes for:
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/15129Co-Authored-By: NAlex Kalderimis <akalderimis@gitlab.com>
Co-Authored-By: NLuke Duncalfe <lduncalfe@eml.cc>