IssueReferenceFilter will end up processing internal issue references
when a project uses an external issues tracker while still using
internal issue references (in the form of `#\d+`). This commit ensures
that these links are rendered as external issue links, regardless of
whether the project one currently views uses an internal or external
issues tracker.

Fixes gitlab-org/gitlab-ce#19036, gitlab-com/performance#16

Add max-height to prevent images from displaying larger than the provided screen size.

Also fix a failing test and add a new one.

This commit changes the way certain documents are rendered (currently
only Notes) and how documents are redacted. Previously both rendering
and redacting would run on a per document basis. The result of this was
that for every document we'd have to run countless queries just to
figure out if we could display a set of links or not.

This commit changes things around so that redacting Markdown documents
is no longer tied into the html-pipeline Gem. This in turn allows it to
redact multiple documents in a single pass, thus reducing the number of
queries needed.

In turn rendering issue/merge request notes has been adjusted to take
advantage of this new setup. Instead of rendering Markdown somewhere
deep down in a view the Markdown is rendered and redacted in the
controller (taking the current user and all that into account). This has
been done in such a way that the "markdown()" helper method can still be
used on its own.

This particular commit also paves the way for caching rendered HTML on
object level. Right now there's an accessor method Note#note_html which
is used for setting/getting the rendered HTML. Once we cache HTML on row
level we can simply change this field to be a column and call a "save"
whenever needed and we're pretty much done.

In the past this class would use Project#get_issue to retrieve an issue
by its ID. This method would automatically determine whether to return
an Issue or ExternalIssue.

This commit changes IssueReferenceFilter to handle external issues again
and in a somewhat more explicit manner than before.

Fixes gitlab-org/gitlab-ce#18827

    A lot of git operations were being repeated, for example, to build a url
    you would ask if the path was a Tree, which would call a recursive routine
    in Gitlab::Git::Tree#where, then ask if the path was a Blob, which would
    call a recursive routine at Gitlab::Git::Blob#find, making reference to
    the same git objects several times. Now we call Rugged::Tree#path, which
    allows us to determine the type of the path in one pass.

    Some other minor improvement added, like saving commonly used references
    instead of calculating them each time.

When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we
get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method
that will throw out out-of-bounds values.

Closes #18777

1. An exception would be raised if the filter was called with an invalid
   URI. Mainly because we weren't catching the `Addressable` exception.

2. This commit fixes it and adds a spec for the filter.

This reduces the number of queries executed in IssueReferenceFilter by
retrieving the various projects/issues that may be referenced in batches
_before_ iterating over all the HTML nodes.

A chunk of the logic resides in AbstractReferenceFilter so it can be
re-used by other filters in the future.

Original Comments
=================

- Linking behaves as per rules documented here:
  https://gitlab.com/gitlab-org/gitlab-ce/blob/16568-document-wiki-linking-behavior/doc/markdown/wiki.md
- All links (to other wiki pages) are rewritten to be at the level of
  the app root. We can't use links relative to the current
  page ('./foo', 'foo', '../foo'), because they won't work in the
  markdown preview, where the current page is suffixed with `/edit`
- Move existing `WikiLinkFilter` specs to `WikiPipeline` spec. It makes
  sense to run these tests on the combined output of the pipeline,
  rather than a single filter, since we can catch issues with
  conflicting filters.
- Add more tests to cover the new linking

@rymai's Review
===============

- Classes nested under `WikiLinkFilter` should declare `WikiLinkFilter`'s
  inherit, so nothing changes if the nested class is loaded first.
- Add a blank line after a guard clause
- Use keyword arguments for the `Rewriter` constructor
- Invert a condition - use `if` instead of `unless`
- Inline a `let` in `WikiPipeline` spec - it was only used in a single place
- Change out of date spec names
- Add a comment for every rewrite rule in `Rewriter`

This reverts commit 3e991230.

# Conflicts:
#	app/models/project.rb

This changes UserReferenceFilter so it operates using the following
steps:

1. Grab all username references from the input document.
2. Query the corresponding Namespace objects using a single query.
3. Iterate over all nodes to build links while re-using the objects
   queried in step 2.

The impact of these changes is that a comment mentioning 5 different
usernames no longer runs 5 different queries (1 for every username),
instead it only runs a single query.

This method returns an Array of the HTML nodes as yielded by
ReferenceFilter#each_node. The method's return value is memoized to
allow multiple calls without having to re-query the input document.

This changes ReferenceFilter#each_node so that when it's called without
a block an Enumerator is returned.

Nokogiri's `node.replace` was being unnecessarily called for every text node in
the document due to a comparison bug. The code previously was comparing the
HTML representation of the full document against the text node, which would
always fail. Fix the comparison to just compare the modified text.

Closes #18011

A user viewing the TODOs page will see a 404 if there are mentioned labels
in multiple different projects. This is likely a caching bug and only occurs
when Markdown rendering occurs across multiple projects, which is why it's so
tricky to reproduce. This is what I think is happening:

1. LabelReferenceFilter#references_in encounters label ~X for ProjectA and finds the label in the DB as id = 1.
2. LabelReferenceFilter.references_in yields [1, 'X', nil, ...]
3. Since project_ref is nil, AbstractReferenceFilter#project_from_ref_cache caches nil => ProjectA.
4. LabelReferenceFilter#references_in encounters label ~Y for ProjectB and finds the label in the DB as id = 2.
5. LabelReferenceFilter.references_in yields [2, 'Y', nil, ...]
6. AbstractReferenceFilter#project_from_ref_cache lookups nil and returns ProjectA. It was supposed to be ProjectB.
7. A is the wrong project, so the label lookup fails.

This MR caches Markdown references if the key is present.

Closes #17898

This splits the Markdown rendering and reference extraction phases into
two distinct code bases. The reference extraction phase no longer relies
on the html-pipeline Gem (and any related code) and allows for
extracting of references from multiple HTML nodes in a single pass. This
means that if you want to extract user references from 200 comments you
no longer need to run 200 times N number of queries, instead only a
handful of queries may be needed.

- And fix behavior for non-file hierarchical links.

Closes #17342, closes #17363

Closes #1625

Closes #15168

Also, addint a suffix to the reference text when the milestone is in another project

Using the syntax proposed in #13829 [project_reference]%(milestone_id | milestone_name)
to get a link to the referred milestone.