Actually, `first` taking non-deterministic result is considered a bug to
me. But changing the behavior should not be happened in rc2.

I've restored the behavior for 6.0, and then will deprecate that in 6.1.

Fixes #36802.

#36805 have one possible regression that failing deduplication if
`joins_values` have complex order (e.g. `joins_values = [join_node_a,
:comments, :tags, join_node_a]`).

This fixes the deduplication to take it in the first phase before
grouping.

Currently, string joins are always applied as last joins part, and Arel
join nodes are always applied as leading joins part (since #36304), it
makes people struggled to preserve user supplied joins order.

To mitigate this problem, preserve the order of string joins and Arel
join nodes either before or after of association joins.

Fixes #36761.
Fixes #34328.
Fixes #24281.
Fixes #12953.

GROUP BY with virtual count attribute is invalid for almost all
databases, but it is valid for PostgreSQL, and it had worked until Rails
5.2.2, so it is a regression for Rails 5.2.3 (caused by 311f0011).

I can't find perfectly solution for fixing this for now, but I would not
like to break existing apps, so I decided to allow referencing virtual
count attribute in ORDER BY clause when GROUP BY aggrigation (it partly
revert the effect of 311f0011) to fix the regression #36022.

Fixes #36022.

`split(/\s*,\s*/)` to order args and then `permit.match?` one by one is
much slower than `permit.match?` once.

Currently `posts.title` is regarded as a safe SQL string, but
`"posts"."title"` (it is a result of `quote_table_name("posts.title")`)
is regarded as an unsafe SQL string even though a result of
`quote_table_name` should obviously be regarded as a safe SQL string,
since the column name matcher doesn't respect quotation, it is a little
annoying.

This changes the column name matcher to allow quoted identifiers as safe
SQL string, now all results of the `quote_table_name` are regarded as
safe SQL string.

#36293 was an issue for through association with `joins` for a long
time, but since #35864 through association with `left_joins` would also
be affected by the issue.

Implicit through table joins should be appeared before user supplied
joins, otherwise loading through association with joins will cause a
statement invalid error.

Fixes #36293.

```
% ARCONN=postgresql bundle exec ruby -w -Itest test/cases/associations/has_many_through_associations_test
.rb -n test_through_association_with_joins
Using postgresql
Run options: -n test_through_association_with_joins --seed 7116

# Running:

E

Error:
HasManyThroughAssociationsTest#test_through_association_with_joins:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  missing FROM-clause entry for table "posts"
LINE 1: ... "comments_posts" ON "comments_posts"."post_id" = "posts"."i...
                                                             ^
: SELECT "comments".* FROM "comments" INNER JOIN "comments" "comments_posts" ON "comments_posts"."post_id" = "posts"."id" INNER JOIN "posts" ON "comments"."post_id" = "posts"."id" WHERE "posts"."author_id" = $1

rails test test/cases/associations/has_many_through_associations_test.rb:61

Finished in 0.388657s, 2.5730 runs/s, 0.0000 assertions/s.
1 runs, 0 assertions, 0 failures, 1 errors, 0 skips
```

This fixes a regression for #35864.

Usually, stashed joins (mainly eager loading) are performed as LEFT
JOINs.
But the case of merging joins/left_joins of different class, that
(stashed) joins are performed as the same `join_type` as the parent
context for now.
Since #35864, both (joins/left_joins) stashed joins might be contained
in `joins_values`, so each stashed joins should maintain its own
`join_type` context.

Fixes #36103.

If we want to get alias resolved attribute finally, we can use
`attribute_alias` directly.

For that purpose, avoiding redundant `attribute_alias?` makes alias
attribute access 40% faster.

https://gist.github.com/kamipo/e427f080a27b46f50bc508fae3612a0e

Before (2c0729d8):

```
Warming up --------------------------------------
          user['id']   102.668k i/100ms
      user['new_id']    80.660k i/100ms
        user['name']    99.368k i/100ms
    user['new_name']    81.626k i/100ms
Calculating -------------------------------------
          user['id']      1.431M (± 4.0%) i/s -      7.187M in   5.031985s
      user['new_id']      1.042M (± 4.2%) i/s -      5.243M in   5.039858s
        user['name']      1.406M (± 5.6%) i/s -      7.055M in   5.036743s
    user['new_name']      1.074M (± 3.6%) i/s -      5.387M in   5.024152s
```

After (this change):

```
Warming up --------------------------------------
          user['id']   109.775k i/100ms
      user['new_id']   103.303k i/100ms
        user['name']   105.988k i/100ms
    user['new_name']    99.618k i/100ms
Calculating -------------------------------------
          user['id']      1.520M (± 6.7%) i/s -      7.574M in   5.011496s
      user['new_id']      1.485M (± 6.2%) i/s -      7.438M in   5.036252s
        user['name']      1.538M (± 5.4%) i/s -      7.737M in   5.049765s
    user['new_name']      1.516M (± 4.6%) i/s -      7.571M in   5.007293s
```

Those helper methods makes relation values access 15% slower.

https://gist.github.com/kamipo/e64439f7a206e1c5b5c69d92d982828e

Before (02b5b8cb):

```
Warming up --------------------------------------
        #limit_value   237.074k i/100ms
    #limit_value = 1   222.052k i/100ms
Calculating -------------------------------------
        #limit_value      6.477M (± 2.9%) i/s -     32.479M in   5.019475s
    #limit_value = 1      5.297M (± 4.3%) i/s -     26.424M in   4.999933s
```

After (this change):

```
Warming up --------------------------------------
        #limit_value   261.109k i/100ms
    #limit_value = 1   239.646k i/100ms
Calculating -------------------------------------
        #limit_value      7.412M (± 1.6%) i/s -     37.077M in   5.003345s
    #limit_value = 1      6.134M (± 1.0%) i/s -     30.675M in   5.000908s
```

`where.not` with polymorphic association is partly fixed incidentally at
213796fb (refer #33493, #26207, #17010, #16983, #14161), and I've added
test case e9ba12f7 to avoid lose that fix accidentally in the future.

In Rails 5.2, `where.not(polymorphic: object)` works as expected as
NAND, but `where.not(polymorphic_type: object.class.polymorphic_name,
polymorphic_id: object.id)` still unexpectedly works as NOR.

To will make `where.not` working desiredly as NAND in Rails 6.1, this
deprecates `where.not` working as NOR. If people want to continue NOR
conditions, we'd encourage to them to `where.not` each conditions
manually.

```ruby
all = [treasures(:diamond), treasures(:sapphire), cars(:honda), treasures(:sapphire)]
assert_equal all, PriceEstimate.all.map(&:estimate_of)
```

In Rails 6.0:

```ruby
sapphire = treasures(:sapphire)

nor = all.reject { |e|
  e.estimate_of_type == sapphire.class.polymorphic_name
}.reject { |e|
  e.estimate_of_id == sapphire.id
}
assert_equal [cars(:honda)], nor

without_sapphire = PriceEstimate.where.not(
  estimate_of_type: sapphire.class.polymorphic_name, estimate_of_id: sapphire.id
)
assert_equal nor, without_sapphire.map(&:estimate_of)
```

In Rails 6.1:

```ruby
sapphire = treasures(:sapphire)

nand = all - [sapphire]
assert_equal [treasures(:diamond), cars(:honda)], nand

without_sapphire = PriceEstimate.where.not(
  estimate_of_type: sapphire.class.polymorphic_name, estimate_of_id: sapphire.id
)
assert_equal nand, without_sapphire.map(&:estimate_of)
```

Resolves #31209.

This is also resolved in `arel_column`.

Originally the `JoinDependency` has the deduplication for eager loading
(LEFT JOIN). This re-uses that deduplication for `left_joins`.

And also, This makes left join order into part of joins, i.e.:

Before:

```
association joins -> stash joins (eager loading, etc) -> string joins -> left joins
```

After:

```
association joins -> stash joins (eager loading, left joins, etc) -> string joins
```

Now string joins are able to refer left joins.

Fixes #34325.
Fixes #34332.
Fixes #34536.

I've experienced this issue in our app, some hints only works on Top
level query (e.g. `MAX_EXECUTION_TIME`).

* Add `ActiveRecord::Relation#extract_associated` for extracting associated records from a relation

Update doc for #includes to clarify that #references is unnecessary when conditions are passed into #includes as a hash.

This patch has two main portions:

1. Add SQL comment support to Arel via Arel::Nodes::Comment.
2. Implement a Relation#annotate method on top of that.

== Adding SQL comment support

Adds a new Arel::Nodes::Comment node that represents an optional SQL
comment and teachers the relevant visitors how to handle it.

Comment nodes may be added to the basic CRUD statement nodes and set
through any of the four (Select|Insert|Update|Delete)Manager objects.

For example:

    manager = Arel::UpdateManager.new
    manager.table table
    manager.comment("annotation")
    manager.to_sql # UPDATE "users" /* annotation */

This new node type will be used by ActiveRecord::Relation to enable
query annotation via SQL comments.

== Implementing the Relation#annotate method

Implements `ActiveRecord::Relation#annotate`, which accepts a comment
string that will be appeneded to any queries generated by the relation.

Some examples:

    relation = Post.where(id: 123).annotate("metadata string")
    relation.first
    # SELECT "posts".* FROM "posts" WHERE "posts"."id" = 123
    # LIMIT 1 /* metadata string */

    class Tag < ActiveRecord::Base
      scope :foo_annotated, -> { annotate("foo") }
    end
    Tag.foo_annotated.annotate("bar").first
    # SELECT "tags".* FROM "tags" LIMIT 1 /* foo */ /* bar */

Also wires up the plumbing so this works with `#update_all` and
`#delete_all` as well.

This feature is useful for instrumentation and general analysis of
queries generated at runtime.

We as Arm Treasure Data are using Optimizer Hints with a monkey patch
(https://gist.github.com/kamipo/4c8539f0ce4acf85075cf5a6b0d9712e),
especially in order to use `MAX_EXECUTION_TIME` (refer #31129).

Example:

```ruby
class Job < ApplicationRecord
  default_scope { optimizer_hints("MAX_EXECUTION_TIME(50000) NO_INDEX_MERGE(jobs)") }
end
```

Optimizer Hints is supported not only for MySQL but also for most
databases (PostgreSQL on RDS, Oracle, SQL Server, etc), it is really
helpful to turn heavy queries for large scale applications.

#35360 allows table name qualified if `from` has original table name.
But that is still too strict. We have a valid use case that `from` with
INDEX hint (e.g. `from("comments USE INDEX (PRIMARY)")`).
So I've relaxed the table name detection in `from` to allow any
extension like INDEX hint.

Fixes #35359.

This is caused by 0ee96d13.

Since #18744, `select` columns doesn't be qualified by table name if
using `from`. 0ee96d13 follows that for `pluck` as well.

But people depends that `pluck` columns are qualified even if using
`from`.

So I've fixed that to be qualified if `from` has the original table name
to keep the behavior as much as before.

Fixes #35359.

Revert "Fix lint `ShadowingOuterLocalVariable`"

This reverts commit 38bd45a48992b500478a82d56d31468a322937a8.

Change of variable name

Fix lint `ShadowingOuterLocalVariable`

This follows up 0ee96d13.

Currently custom attributes are always qualified by the table name in
the generated SQL wrongly even if the table doesn't have the named
column, it would cause an invalid SQL error.

Custom attributes should only be qualified if the table has the same
named column.

Since Rails 6.0 will support Ruby 2.4.1 or higher
`# frozen_string_literal: true` magic comment is enough to make string object frozen.
This magic comment is enabled by `Style/FrozenStringLiteralComment` cop.

* Exclude these files not to auto correct false positive `Regexp#freeze`
 - 'actionpack/lib/action_dispatch/journey/router/utils.rb'
 - 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb'

It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333
Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed.

* Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required

 - 'actionpack/test/controller/test_case_test.rb'
 - 'activemodel/test/cases/type/string_test.rb'
 - 'activesupport/lib/active_support/core_ext/string/strip.rb'
 - 'activesupport/test/core_ext/string_ext_test.rb'
 - 'railties/test/generators/actions_test.rb'

Currently, column aliases which is used for eager loading are calculated
before constructing all table aliases in FROM clause.

`JoinDependency#join_constraints` constructs table aliases for `joins`
first, and then always re-constructs table aliases for eager loading.

If both `joins` and eager loading are given a same table association,
the re-construction would cause the discrepancy between column aliases
and table aliases.

To avoid the discrepancy, the column aliases should be calculated after
all table aliases are constructed.

Fixes #30603.

Adds test case for failing issue

Moves set_value back to protected