Fix export files not removed when a user takes over a namespace

See merge request !2051

This adds counters for build artifacts and LFS objects, and moves
the preexisting repository_size and commit_count from the projects
table into a new project_statistics table.

The counters are displayed in the administration area for projects
and groups, and also available through the API for admins (on */all)
and normal users (on */owned)

The statistics are updated through ProjectCacheWorker, which can now
do more granular updates with the new :statistics argument.

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Currently namespace name and path have uniq validaiton which does not
allow us to use same group name/path inside different groups. This
commit changes validation in next way:

* Allow same namespace name with different parent_id
* Allow same namespace path. Uniq validation should be handled by routes table
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

* add parent_id field to namespaces table to store relation with nested groups
* create routes table to keep information about full path of every group and project
* project/group lookup by full path from routes table
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NRémy Coutable <remy@rymai.me>

Adds .git and .atom to the master namespace regex
Updates existing group tests and adds two new ones
Updates path cleaning to also forbid .atom

This commit adds a number of _html columns and, with the exception of Note,
starts updating them whenever the content of their partner fields changes.

Note has a collision with the note_html attr_accessor; that will be fixed later

A background worker for clearing these cache columns is also introduced - use
`rake cache:clear` to set it off. You can clear the database or Redis caches
separately by running `rake cache:clear:db` or `rake cache:clear:redis`,
respectively.

Groups can enable/disable LFS, but this setting can be overridden at the project level. Admin only

There is a race condition in DestroyGroupService now that projects are deleted asynchronously:

1. User attempts to delete group
2. DestroyGroupService iterates through all projects and schedules a Sidekiq job to delete each Project
3. DestroyGroupService destroys the Group, leaving all its projects without a namespace
4. Projects::DestroyService runs later but the can?(current_user,
   :remove_project) is `false` because the user no longer has permission to
   destroy projects with no namespace.
5. This leaves the project in pending_delete state with no namespace/group.

Projects without a namespace or group also adds another problem: it's not possible to destroy the container
registry tags, since container_registry_path_with_namespace is the wrong value.

The fix is to destroy the group asynchronously and to run execute directly on Projects::DestroyService.

Closes #17893

This reverts commit 3e991230.

# Conflicts:
#	app/models/project.rb

In 8278b763 the default behaviour of annotation
has changes, which was causing a lot of noise in diffs. We decided in #17382
that it is better to get rid of the whole annotate gem, and instead let people
look at schema.rb for the columns in a table.

Fixes: #17382

This ensures searching namespaces works exactly the same as searching
for any other resource.

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

This reverts commit 5daf44b7, reversing
changes made to 2f706fbd.

This reverts commit 957e849f, reversing
changes made to 6b9dbe9f.
Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

This reverts commit c0116926.