Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

Signed-off-by: NRémy Coutable <remy@rymai.me>

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10901/diffs
introduced a new naming scheme for backups, but the code which
cleans up old backups wasn't updated accordingly. In order to
maintain backward compatibility, we need to account for 3 naming
schemes.

Instead of doing hacks like http://stackoverflow.com/a/26082612/974710

Since version ruby-2.2.0, method `File::exists?` is deprecated.

Closes #12710

Colorize is a gem licensed under the GPLv2, so we can’t use it in GitLab without relicensing GitLab under the terms of the GPL. Rainbow is licensed under the MIT license and does the exact same thing as Colorize, so Rainbow was added in place of Colorize.

The syntax is slightly different for Rainbow vs. Colorize, and was updated in accordance.

The gem is still a dependency of Spinach, so it’s included in the development/test environments, but won’t be packaged with the actual product, and therefore doesn’t require we relicense the product.

An attempt at relicensing Colorize was made, but didn’t succeed as the library owner never responded.

Rainbow library: https://github.com/sickill/rainbow
Relevant issue regarding licensing in GitLab's gems: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3775

This reverts commit 3e991230.

# Conflicts:
#	app/models/project.rb

This is idempotent, so there's no harm calling it if the directory
already exists.

Closes #12710

- Offloads uploading to GitLab Workhorse
- Use /authorize request for fast uploading
- Added backup recipes for artifacts
- Support download acceleration using X-Sendfile

Documentation elsewhere refers to this internal path, let's keep
it.

By using light gzip compression we can save a lot of disk IO during
the backup.

During the backup we create an intermediate copy of two directories:
builds and uploads. Instead of creating many small files with 'cp
-r', we now use tar (and fast gzip) to create single intermediate
files. This saves on disk IO and disk space while creating a backup.

This adds support for AWS S3 SSE with S3 managed keys, this means the
data is encrypted at rest and the encryption is handled transparently to
the end user as well as in the AWS Console. This is optional and not
required to make S3 uploads work.

This sidesteps problems with running 'chmod' on some CIFS mounts.

This change helps system administrators who want to replicate
GitLab backup files without needing root permissions.