When I proposed using serializable transactions I was hoping we would be
able to refresh data of individual users concurrently. Unfortunately
upon closer inspection it was revealed this was not the case. This could
result in a lot of queries failing due to serialization errors,
overloading the database in the process (given enough workers trying to
update the target table).

To work around this we're now using a Redis lease that is cancelled upon
completion. This ensures we can update the data of different users
concurrently without overloading the database.

The code will try to obtain the lease until it succeeds, waiting at
least 1 second between retries. This is necessary as we may otherwise
end up _not_ updating the data which is not an option.

Signed-off-by: NDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Closes #23150

This method can be used to retrieve a list of projects for a user that
said user has reporter access to. This list is then be reduced down to
a specific set of projects. This allows you to reduce a list of projects
to a list of projects you have reporter access to in an efficient
manner.

It is not possible to search for a user by his secondary email address in
the Users search bar in the admin interface(/admin/users). A use-case could
 be that an admin wants to remove a specific secondary email address of an
user, because it interferes with another user. Issue #23761

This commit adds ability to search not only by main email, but also
by any secondary email in the admin interface.

They were Rails' default and are unnecessarily overridden.
Signed-off-by: NDavid Wagner <david@marvid.fr>

This refactors Gitlab::Identifier so it uses fewer queries and is
actually tested. Queries are reduced by caching the output as well as
using 1 query (instead of 2) to find a user using an SSH key.

Use just SQL to check is a user can admin_issue on a project

Tradeoff

- we duplicate how we check admin_issue in a SQL relation in the Ability class

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5620#note_13671791

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5620#note_13540099

Prior, when providing a `min_access_level` parameter to this method, we
called `Gitlab::Access.values` instead of `all_values`, mistakenly
omitting the `OWNER` level.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/19878

Refactor and rename `restricted_signup_domains` to `domain_whitelist` to better conform to its behavior and newly introduced behavior.

As requested by the issue #14508 this adds an option in the application
settings to set newly registered users by default as external. The
default setting is set to false to stay backward compatible.

The issue was with the `User#groups` and `User#projects` associations
which goes through the `User#group_members` and `User#project_members`.

Initially I chose to use a secure approach by storing the requester's
user ID in `Member#created_by_id` instead of `Member#user_id` because I
was aware that there was a security risk since I didn't know the
codebase well enough.

Then during the review, we decided to change that and directly store the
requester's user ID into `Member#user_id` (for the sake of simplifying
the code I believe), meaning that every `group_members` / `project_members`
association would include the requesters by default...

My bad for not checking that all the `group_members` / `project_members`
associations and the ones that go through them (e.g. `Group#users` and
`Project#users`) were made safe with the `where(requested_at: nil)` /
`where(members: { requested_at: nil })` scopes.

Now they are all secure.
Signed-off-by: NRémy Coutable <remy@rymai.me>

Feedback from:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12301563

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12274602

Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12217415

Feedback from:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12194552

Feedback from:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12194489

Feedback from:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4404#note_12194471

- To hold registrations from U2F devices, and to authenticate them.
- Previously, `User#two_factor_enabled` was aliased to the
  `otp_required_for_login` column on `users`.
- This commit changes things a bit:
    - `User#two_factor_enabled` is not a method anymore
    - `User#two_factor_enabled?` checks both the
      `otp_required_for_login` column, as well as `U2fRegistration`s
    - Change all instances of `User#two_factor_enabled` to
      `User#two_factor_enabled?`
- Add the `u2f` gem, and implement registration/authentication at the
  model level.

This reverts commit 3e991230.

# Conflicts:
#	app/models/project.rb