Fixes #30315.

This is the fix for the regression of #29848.

In #29848, I've kept existing select list in the subquery for the count
if ORDER BY is given. But it had accidentally affect to GROUP BY
queries also. It should keep the previous behavior in that case.

Fixes #30886.

I don't think this is a good abstraction because the internal method is
used only if the relation need to be applied join dependency.

When using a has_many through relation and then summing an attribute
the distinct was not being used. This will ensure that when summing
an attribute, the number is only used once when distinct has been used.

A common source of bugs and code bloat within Active Record has been the
need for us to maintain the list of bind values separately from the AST
they're associated with. This makes any sort of AST manipulation
incredibly difficult, as any time we want to potentially insert or
remove an AST node, we need to traverse the entire tree to find where
the associated bind parameters are.

With this change, the bind parameters now live on the AST directly.
Active Record does not need to know or care about them until the final
AST traversal for SQL construction. Rather than returning just the SQL,
the Arel collector will now return both the SQL and the bind parameters.
At this point the connection adapter will have all the values that it
had before.

A bit of this code is janky and something I'd like to refactor later. In
particular, I don't like how we're handling associations in the
predicate builder, the special casing of `StatementCache::Substitute` in
`QueryAttribute`, or generally how we're handling bind value replacement
in the statement cache when prepared statements are disabled.

This also mostly reverts #26378, as it moved all the code into a
location that I wanted to delete.

/cc @metaskills @yahonda, this change will affect the adapters

Fixes #29766.
Fixes #29804.
Fixes #26541.
Close #28539.
Close #24769.
Close #26468.
Close #26202.

There are probably other issues/PRs that can be closed because of this
commit, but that's all I could find on the first few pages.

Since #26972, `ORDER BY` is kept if `LIMIT` is presented for
performance. But in most SQL servers (e.g. PostgreSQL, SQL Server, etc),
`ORDER BY` expressions must appear in select list for `SELECT DISTINCT`.
We should not replace existing select list in that case.

The `find_each`, `find_in_batches` and `in_batches` APIs usually operate
on large numbers of records, where it's preferable not to load them all
into memory at once.

If the query cache is enabled, it will hold onto the query results until
the end of the execution context (request/job), which means the memory
used is still proportional to the total number of records. These queries
are typically not repeated, so the query cache isn't desirable here.

This reverts commit 3420a145, reversing
changes made to afb66a5a.

The delegation was needed since passing `relation` with
`relation.bound_attributes`. It should use `relation.arel` in that case.

Deprecate passing arguments and block at the same time to `count` and `sum` in `ActiveRecord::Calculations`

`select`, `count`, and `sum` in `Relation` are also `Enumerable` method
that can be passed block. `select` with block already doesn't take
arguments since 4fc3366d. This is follow up of that.

The argument of `Arel::SelectManager.new` is `table`, not `engine`.

https://github.com/rails/arel/blob/v8.0.0/lib/arel/select_manager.rb#L10

Follow up of #24203.

Since b644964b `ActiveRecord::Relation` includes `Enumerable` so it is
enough to call `super` simply.

When a grouped calculation contains a having clause that references a
selected value, we need to include that selected value in the query.

Postgres doesn't support referencing a selected value in a having
clause, but other databases do; we can skip the test on the pg adapter
but run it for the others.

This was fixed before in 9a298a16, but
the test coverage was lost in 5a05207d.
The fix regressed in 6311975f and was
removed in 97d46c17.

`select_values` is a local variable defined at previous line.
`select_values += select_values` is totally useless.

`DISTINCT` clause is applied inside aggregate function by
`operation_over_aggregate_column` if needed. Unneeded outside aggregate
function.

```ruby
  # Before
  author.unique_categorized_posts.count
  # => SELECT DISTINCT COUNT(DISTINCT "posts"."id") FROM "posts" INNER JOIN "categorizations" ON "posts"."id" = "categorizations"."post_id" WHERE "categorizations"."author_id" = ?  [["author_id", 2]]

  # After
  author.unique_categorized_posts.count
  # => SELECT COUNT(DISTINCT "posts"."id") FROM "posts" INNER JOIN "categorizations" ON "posts"."id" = "categorizations"."post_id" WHERE "categorizations"."author_id" = ?  [["author_id", 2]]
```

Closes #27615

This reverts commit 28977f1f.

If select clause is specified and last column has a column alias,
additional column alias causes a statement invalid.

Add test coverage for counting a single column with NULL values.

Fixes #27676, #27682, and #27705.

Actually, private methods cannot be called with `self.`, so it's not just redundant, it's a bad habit in Ruby

If `limit_value` is presented, records fetching order is very important
for performance. Should not unscope the order in the case.

All indentation was normalized by rubocop auto-correct at 80e66cc4.
But comments was still kept absolute position. This commit aligns
comments with method definitions for consistency.

`column_for` was changed to `type_for` to return `type` object at
36bd52b4. But variable name is still `column`. It is very confusing.
Rename variable name `column` to `type` for readability.

For reduce instantiating `Type::Value`.

The current behaviour of checking if there is a LEFT OUTER JOIN arel
node to detect if we are doing eager_loading is wrong. This problem
wasn't frequent before as only some pretty specific cases would add
a LEFT OUTER JOIN arel node. However, the recent new feature
left_outer_joins also add this node and made this problem happen
frequently.

Since in the perform_calculation function, we don't have access to
eager_loading information, I had to extract the logic for the distinct
out to the calculate method.

As I was in the file for left_outer_join tests, I fixed a few that had
bugs and I replaced some that were really weak with something that
will catch more issues.

In relation tests, the first test I changed would have failed if it
had validated the hash returned by count instead of just checking how
many pairs were in it. This is because this merge of join currently
transforms the join node into an outer join node, which then made
count do a distinct. So before this change, the return was
{1=>1, 4=>1, 5=>1}.

The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.

When `group` is used in combination with any calculation method, the
resulting hash uses the grouping expression as the key. Currently we're
incorrectly always favoring the type reported by the query, instead of
the type known by the class. This causes differing behavior depending on
whether the adaptor actually gives proper types with the query or not.
After this change, the behavior will be the same on all adaptors -- we
see if we know the type from the class, fall back to the type from the
query, and finally fall back to the identity type.

Fixes #25595

Ruby 2.4 unifies Fixnum and Bignum into Integer: https://bugs.ruby-lang.org/issues/12005

* Forward compat with new unified Integer class in Ruby 2.4+.
* Backward compat with separate Fixnum/Bignum in Ruby 2.2 & 2.3.
* Drops needless Fixnum distinction in docs, preferring Integer.

This still isn't as separated as I'd like, but it at least moves most of
the burden of alias mapping in one place.

* When tried to use `Company#accounts` test/models/company.rb I got:

```
ActiveRecord::StatementInvalid: SQLite3::SQLException: no such column:
accounts.company_id: SELECT COUNT(*) AS count_all, "companies"."firm_id"
AS companies_firm_id FROM "companies" INNER JOIN "accounts" ON
"accounts"."company_id" = "companies"."id" GROUP BY "companies"."firm_id"
```

* The refactor on Calculations class was just to simplify the code