Addresses: Issue #13810

1. Adds a last_used_at attribute to the Key table/model
2. Update a key's last_used_at whenever it gets used
3. Display how long ago an ssh key was last used

This column used to be a 32 bits integer, allowing for only a maximum of
2 147 483 647 rows. Given enough users one can hit this limit pretty
quickly, as was the case for GitLab.com.

Changing this type to bigint (= 64 bits) would give us more space, but
we'd eventually hit the same limit given enough users and projects. A
much more sustainable solution is to simply drop the "id" column.

There were only 2 lines of code depending on this column being present,
and neither truly required it to be present. Instead the code now uses
the "project_id" column combined with the "user_id" column. This means
that instead of something like this:

    DELETE FROM project_authorizations
    WHERE user_id = X
    AND id = Y;

We now run the following when removing rows:

    DELETE FROM project_authorizations
    WHERE user_id = X
    AND project_id = Y;

Since both user_id and project_id are indexed this should not slow down
the DELETE query.

This commit also removes the "dependent: destroy" clause from the
"project_authorizations" relation in the User and Project models.
Keeping this prevents Rails from being able to remove data as it relies
on an "id" column being present. Since the "project_authorizations"
table has proper foreign keys set up (with cascading removals) we don't
need to depend on any Rails logic.

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Revert MattermostNotificationService and SlackNotificationService to MattermostService and SlackService

On GitLab.com this migration could take about 3 minutes. Disabling the
statement we know we have enough time to complete this.

Fixes #25976

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

This return is redundant as the query now uses "WHERE EXISTS (...)" to
filter out projects without a namespace.

This ensures threads don't re-use the same connection object, and use
fewer queries to perform the work.

We cant have project with name 'project' or 'tree' anymore.
This merge request containts a migration that will find and rename all
projects using reserved names by adding N digit to the end of the name.
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

This adds counters for build artifacts and LFS objects, and moves
the preexisting repository_size and commit_count from the projects
table into a new project_statistics table.

The counters are displayed in the administration area for projects
and groups, and also available through the API for admins (on */all)
and normal users (on */owned)

The statistics are updated through ProjectCacheWorker, which can now
do more granular updates with the new :statistics argument.

This adds a migration to remove unused services, where the properties
are empty. As the properties are empty, those do not contain any
settings or other information.

Fixes #25727

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

If we leave this as a regular migration, we could have the following flow:

1. Application knows nothing about scopes.
2. First migration runs, all existing personal access tokens have `api` scope
3. Application still knows nothing about scopes.
4. Second migration runs, all tokens created after this point have no scope
5. Application still knows nothing about scopes.
6. Tokens created at this time _should have the API scope, but instead have no scope_
7. Application code is reloaded, application knows about scopes
8. Tokens created after this point only have no scope if the user deliberately
   chooses to have no scopes.

Point #6 is the problem here. To avoid this, we move the second migration to a
"post" migration, which runs after the application code is deployed/reloaded.

- Mainly whitespace changes.

- Require the migration adding the `scope` column to the
  `personal_access_tokens` table to have downtime, since API calls will
  fail if the new code is in place, but the migration hasn't run.

- Minor refactoring - load `@scopes` in a `before_action`, since we're
  doing it in three different places.

If the source encoding is not UTF-8 we need to encode the data as
`JSON.dump` may throw an error if the input can not be converted to
UTF-8. We only encode when necessary to reduce the overhead.

Fixes gitlab-org/gitlab-ce#25489

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Currently namespace name and path have uniq validaiton which does not
allow us to use same group name/path inside different groups. This
commit changes validation in next way:

* Allow same namespace name with different parent_id
* Allow same namespace path. Uniq validation should be handled by routes table
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

* add parent_id field to namespaces table to store relation with nested groups
* create routes table to keep information about full path of every group and project
* project/group lookup by full path from routes table
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

This is useful for when runner is bundled with gitlab, like in a kubernetes stack, and we want the runner to be able to
register with gitlab as soon as they both come up.

By passing commit data to this worker we remove the need for querying
the Git repository for every job. This in turn reduces the time spent
processing each job.

The migration included migrates jobs from the old format to the new
format. For this to work properly it requires downtime as otherwise
workers may start producing errors until they're using a newer version
of the worker code.

Signed-off-by: NRémy Coutable <remy@rymai.me>

This new global setting will allow admins to specify if HTML emails should be sent or not,
this is basically useful when system administrators want to save some disk space by avoiding
emails in HTML format and using only the Plain Text version.

This is to ensure that migrations will still be consitent when we will
upgrade to Rails 5 which default to `null: false` for timestamps
columns.

Fixes #23666.
Signed-off-by: NRémy Coutable <remy@rymai.me>

When I proposed using serializable transactions I was hoping we would be
able to refresh data of individual users concurrently. Unfortunately
upon closer inspection it was revealed this was not the case. This could
result in a lot of queries failing due to serialization errors,
overloading the database in the process (given enough workers trying to
update the target table).

To work around this we're now using a Redis lease that is cancelled upon
completion. This ensures we can update the data of different users
concurrently without overloading the database.

The code will try to obtain the lease until it succeeds, waiting at
least 1 second between retries. This is necessary as we may otherwise
end up _not_ updating the data which is not an option.