To be able to check regardless of locale.

Enables the query cache on the correct connection when
shared connections across threads are enabled

fixes #36610

It appears that Oracle does not allow using aliases in GROUP BY clause
unlike ORDER BY clause.

Fixes #36613.

The PostgreSQL adapter uses an error message to determine if a database
exists or not.
https://github.com/rails/rails/blob/74ef67b16de67d2ae2f996e50a18a93aebf68fe6/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb#L49

However, this message is properly converted according to the locale.
So this check does not work correctly for non-en locales.

As a result, `db:prepare` cannot correctly determine if a database exists, and
`bin/setup`, which depends on the task, does not work correctly if the database
does not exist.

It checks to exist if the "does not exist" exists, but that message is also
used in other error messages(e.g. "role does not exist"). So cannot check
correctly also in en locale.
https://github.com/postgres/postgres/blob/master/src/backend/po/ja.po#L10542

It would be fine could check the status, but in my understanding, when a connecting
fails, only the status `CONNECTION_BAD` be used, and it seems that details cannot
be checked.
https://www.postgresql.org/docs/11/libpq-status.html#LIBPQ-PQSTATUS

I fixed to check whether the error message contains a database
name. This is probably not accurate but can check it better now.

`enum` and `set` are typed cast as `:string`, but currently the
`:string` type is incorrectly reused for schema dumping.

A cast type on columns is not always the same with `sql_type`, this
fixes schema dumping `enum` and `set` columns to use `sql_type` instead
of `type` correctly.

This makes to be able to ignore the query in `assert_queries` even if
accidentally reconnected a connection.

https://buildkite.com/rails/rails/builds/61917#4c49187a-3173-4d5c-8a8d-d65768f5bfc9/1000-1799

Introduced in bba7c63a

Before:

```
$ TESTOPTS="-n=/test_yaml_dump_and_load/" bundle exec rake
test:postgresql

:scisors: ... :scisors:

Using postgresql
Run options: -n=/test_yaml_dump_and_load/ --seed 36896

/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
/home/u/code/rails/activerecord/lib/active_record/connection_adapters/postgresql/column.rb:15:
warning: instance variable @serial not initialized
.

Finished in 0.195325s, 5.1197 runs/s, 35.8376 assertions/s.
1 runs, 7 assertions, 0 failures, 0 errors, 0 skips

```
Co-authored-by: NRyuta Kamizono <kamipo@gmail.com>

This PR is to fix #36559 but I also found other issues that haven't been
reported.

The check for `(config.size == 1 && config.values.all? { |v| v.is_a?
String })` was naive. The only reason this passed was because we had
tests that had single hash size configs, but that doesn't mean we don't
want to create a hash config in other cases. So this now checks for
`config["database"] || config["adapter"] || ENV["DATABASE_URL"]`. In the
end for url configs we still get a UrlConfig but we need to pass through
the HashConfig to create the right kind of UrlConfig. The UrlConfig's
are really complex and I don't necessarily understand everything that's
needed in order to act the same as Rails 5.2.

I edited the connection handler test to demonstrate how the previous
implementation was broken when checking config size. Now old and new
tests pass so I think this is closer to 5.2.

Fixes #36559

For multiple databases we attempt to generate the tasks by reading the
database.yml before the Rails application is booted. This means that we
need to strip out ERB since it could be reading Rails configs.

In some cases like https://github.com/rails/rails/issues/36540 the ERB
is too complex and we can't overwrite with the DummyCompilier we used in
https://github.com/rails/rails/pull/35497. For the complex causes we
simply issue a warning that says we couldn't infer the database tasks
from the database.yml.

While working on this I decided to update the code to only load the
database.yml once initially so that we avoid having to issue the same
warning multiple times. Note that this had no performance impact in my
testing and is merely for not having to save the error off somewhere.
Also this feels cleaner.

Note that this will not break running tasks that exist, it will just
mean that tasks for multi-db like `db:create:other_db` will not be
generated. If the database.yml is actually unreadable it will blow up
during normal rake task calls.

Fixes #36540

I think we should change this, but not in 6-0-stable since that's
already in RC and I was trying to only make changes that won't require
any app changes.

This reverts a portion of https://github.com/rails/rails/pull/36439 that
made all schema migration version numbers get dumped as an integer.
While it doesn't _really_ matter it did change behavior. We should bring
this back in 6.1 with a deprecation.

When a record does not have a table name, as in the case for a record
with `self.abstract_class = true` and no `self.table_name` set the error
message raises a cryptic:
"ActiveRecord::StatementInvalid: Could not find table ''" this patch now
raises a new `TableNotSpecified Error`

Fixes: #36274
Co-Authored-By: NEugene Kenny <elkenny@gmail.com>

When SQLite connects it will silently create a database if the database does not
exist. This behaviour causes different issues because of inconsistent behaviour
between adapters: #36383, #32914. This commit adds a `database_exists?` method
as a way to check the database without creating it. This is a stepping stone to
fully resolving the above issues.

Currently `type.serialize` and `connection.{quote|type_cast}` for a time
object always does `time.getutc` call regardless of whether it is
already utc time object or not, that duplicated proccess
(`connection.type_cast(type.serialize(time))`) allocates extra/useless
time objects for each type casting.

This avoids that redundant `time.getutc` call if it is already utc time
object. In the case of a model has timestamps (`created_at` and
`updated_at`), it avoids 6,000 time objects allocation for 1,000 times
`model.save`.

```ruby
ObjectSpace::AllocationTracer.setup(%i{path line type})

pp ObjectSpace::AllocationTracer.trace {
  1_000.times { User.create }
}.select { |k, _| k[0].end_with?("quoting.rb", "time_value.rb") }
```

Before (c104bfe4):

```
{["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  203,
  :T_ARRAY]=>[1004, 0, 778, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  220,
  :T_STRING]=>[2, 0, 2, 1, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  209,
  :T_ARRAY]=>[8, 0, 8, 1, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  57,
  :T_ARRAY]=>[4, 0, 4, 1, 1, 0],
 ["~/rails/activemodel/lib/active_model/type/helpers/time_value.rb",
  17,
  :T_DATA]=>[4000, 0, 3096, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  120,
  :T_DATA]=>[2000, 0, 1548, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  126,
  :T_STRING]=>[4000, 0, 3096, 0, 1, 0]}
```

After (this change):

```
{["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  203,
  :T_ARRAY]=>[1004, 0, 823, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  220,
  :T_STRING]=>[2, 0, 2, 1, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  209,
  :T_ARRAY]=>[8, 0, 8, 1, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  57,
  :T_ARRAY]=>[4, 0, 4, 1, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb",
  126,
  :T_STRING]=>[2000, 0, 1638, 0, 1, 0]}
```

GROUP BY with virtual count attribute is invalid for almost all
databases, but it is valid for PostgreSQL, and it had worked until Rails
5.2.2, so it is a regression for Rails 5.2.3 (caused by 311f0011).

I can't find perfectly solution for fixing this for now, but I would not
like to break existing apps, so I decided to allow referencing virtual
count attribute in ORDER BY clause when GROUP BY aggrigation (it partly
revert the effect of 311f0011) to fix the regression #36022.

Fixes #36022.

`Arel::Attributes.for` is no longer used since https://github.com/rails/arel/pull/196.

Each `visit o, collector` allocates one extra array due to
receiving args by splat array.

https://github.com/rails/rails/blob/2c3332cc4c0fa77dbe2e13e8a792f80fbd8f4ad3/activerecord/lib/arel/visitors/visitor.rb#L27-L29

Currently 1,000 times `User.where(id: 1).to_sql` allocates 13,000
arrays in `visitor.accept`. This avoids receiving args by splat array,
it makes `visitor.accept` no array allocation.

```ruby
ObjectSpace::AllocationTracer.setup(%i{path line type})

pp ObjectSpace::AllocationTracer.trace {
  1_000.times { User.where(id: 1).to_sql }
}.select { |k, _| k[2] == :T_ARRAY && k[0].end_with?("visitor.rb", "to_sql.rb") }
```

Before (2c3332cc):

```
{["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  18,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb",
  11,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/visitor.rb",
  12,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  788,
  :T_ARRAY]=>[3000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  794,
  :T_ARRAY]=>[3000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  156,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  443,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  603,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
 ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
  611,
  :T_ARRAY]=>[1000, 0, 0, 0, 0, 0]}
```

After (this change):

```
{}
```

If we put the `while_preventing_writes` on the connection then the
middleware that sends reads to the primary and ensures they can't write
will not work. The `while_preventing_writes` will only be applied to the
connection which it's called on - which in the case of the middleware is
Ar::Base.

This worked fine if you called it directly like
`OtherDbConn.connection.while_preventing_writes` but Rails didn't have a
way of knowing you wanted to call it on all the connections.

The change here moves the `while_preventing_writes` method from the
connection to the handler so that it can block writes to all queries for
that handler. This will apply to all the connections associated with
that handler.

This PR moves the `schema_migration` to `migration_context` so that we
can access the `schema_migration` per connection.

This does not change behavior of the SchemaMigration if you are using
one database. This also does not change behavior of any public APIs.
`Migrator` is private as is `MigrationContext` so we can change these as
needed.

We now need to pass a `schema_migration` to `Migrator` so that we can
run migrations on the right connection outside the context of a rake
task.

The bugs this fixes were discovered while debugging the issues around
the SchemaCache on initialization with multiple database. It was clear
that `get_all_versions` wouldn't work without these changes outside the
context of a rake task (because in the rake task we establish a
connection and change AR::Base.connection to the db we're running on).

Because the `SchemaCache` relies on the `SchemaMigration` information we
need to make sure we store it per-connection rather than on
ActiveRecord::Base.

[Eileen M. Uchitelle & Aaron Patterson]

As `remove_column` can be reversed when a type is provided this example
was not accurate anymore.

Currently 1,000 transactions creates 10,000 objects regardless whether
it is necessary or not.

This makes allocation on demand in transactions, now 1,000 transactions
creates required 5,000 objects only by default.

```ruby
ObjectSpace::AllocationTracer.setup(%i{path line type})

pp ObjectSpace::AllocationTracer.trace {
  1_000.times { User.create }
}.select { |k, _| k[0].end_with?("transaction.rb") }
```

Before (95d038f0):

```
{["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  209,
  :T_HASH]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  210,
  :T_OBJECT]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  210,
  :T_HASH]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  80,
  :T_OBJECT]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  8,
  :T_ARRAY]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  81,
  :T_ARRAY]=>[1000, 0, 715, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  289,
  :T_STRING]=>[1000, 0, 714, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  116,
  :T_ARRAY]=>[1000, 0, 714, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  120,
  :T_ARRAY]=>[1000, 0, 714, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  121,
  :T_HASH]=>[1000, 0, 714, 0, 1, 0]}
```

After (this change):

```
{["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  213,
  :T_HASH]=>[1000, 0, 739, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  214,
  :T_OBJECT]=>[1000, 0, 739, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  214,
  :T_HASH]=>[1000, 0, 739, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  81,
  :T_OBJECT]=>[1000, 0, 739, 0, 1, 0],
 ["~/rails/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb",
  304,
  :T_STRING]=>[1000, 0, 738, 0, 1, 0]}
```

* Make ActiveRecord `ConnectionPool.connections` thread-safe.

ConnectionPool documentation is clear on the need to synchronize
access to @connections but also states that public methods do not
require synchronization. Existing code exposed @connections
directly via attr_reader. The fix uses synchronize() to lock
@connections then returns a copy to the caller using Array.dup().

Includes comments on the connections method that thread-safe access
to the connections array does not imply thread-safety of accessing
methods on the actual connections.

Adds a test-case that modifies the pool using a supported method
in one thread  while a second thread accesses pool.connections.
The test fails without this patch.

Fixes #36465.

* Update activerecord/test/cases/connection_pool_test.rb

[jeffdoering + Rafael Mendonça França]

`table_exists?` is already exist in `ModelSchema`.

https://github.com/rails/rails/blob/5cab344494c340ea82a35b46efa06b94f0b7730b/activerecord/lib/active_record/model_schema.rb#L339-L341

Otherwise `Model.table_exists?` returns the staled cache result.

We sometimes say "✂️ newline after `private`" in a code review (e.g.
https://github.com/rails/rails/pull/18546#discussion_r23188776,
https://github.com/rails/rails/pull/34832#discussion_r244847195).

Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style
`EnforcedStyle: only_before` (https://github.com/rubocop-hq/rubocop/pull/7059).

That cop and enforced style will reduce the our code review cost.

Currently, almost all "Dangerous query method" warnings are false alarm.
As long as almost all the warnings are false alarm, developers think
"Let's ignore the warnings by using `Arel.sql()`, it actually is false
alarm in practice.", so I think we should effort to reduce false alarm
in order to make the warnings valuable.

This allows column name with function (e.g. `length(title)`) as safe SQL
string, which is very common false alarm pattern, even in the our
codebase.

Related 6c82b6c9, 6607ecb2, #36420.

Fixes #32995.

`split(/\s*,\s*/)` to order args and then `permit.match?` one by one is
much slower than `permit.match?` once.

Method added in https://github.com/rails/rails/pull/36416

This PR proposes moving the schema cache from the connection to the pool
so the connection can ask the pool for the cache. In a future PR our
goal is to be able to read the yaml file from the pool so we can get
rid of the `active_record.check_schema_cache_dump` initializer. This
will fix the issues surrounding dumping the schema cache and mulitple
databases.

Why do we want to get rid of the initializer you ask?

Well I was looking at #34449 and trying to make it work for our usecase
and it revealed A LOT of problems. There are a few issues that I will
fix in remaining PRs with SchemaMigration, but there's a big glaring
issue with this initializer.

When you have an application with multiple databases we'll need to loop
through all the configurations and set the schema cache on those
connections. The problem is on initialization we only have one
connection - the one for Ar::Base. This is fine in a single db
application but not fine in multi-db. If we follow the pattern in #34449
and establish a connection to those other dbs we will end up setting the
cache on the _connection object_ rather than on all connections that
connect for that config.

So even though we looped through the configs and assigned the cache the
cache will not be set (or will be set wrong) once the app is booted
because the connection objects after boot are _different_ than the
connection objects we assigned the cache to.

After trying many different ways to set the schema cache `@tenderlove`
and I came to the conclusion that the initializer is problematic, as is
setting the schema cache twice.

This is part 1 to move the cache to the pool so the cache can read from
the schema cache yaml file instead of setting it when initializing the
app.

To do this we have created a `NullPool` that initializes an empty cache. I
put the `get_schema_cache` and `set_schema_cache` in an `AbstractPool`
so we can share code between `ConnectionPool` and `NullPool` instead of
duplicating code.

Now we only need to set the schema_cache on the pool rather than the
connection. In `discard!` we need to unset the connection from the
schema_cache - we still want the cache just not the connection.

Currently `posts.title` is regarded as a safe SQL string, but
`"posts"."title"` (it is a result of `quote_table_name("posts.title")`)
is regarded as an unsafe SQL string even though a result of
`quote_table_name` should obviously be regarded as a safe SQL string,
since the column name matcher doesn't respect quotation, it is a little
annoying.

This changes the column name matcher to allow quoted identifiers as safe
SQL string, now all results of the `quote_table_name` are regarded as
safe SQL string.

When someone has a multi-db application their `ApplicationRecord` will
look like:

```ruby
class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  connects_to database: { writing: :primary, reading: :replica }
end
```

This will cause us to open 2 connections to ActiveRecord::Base's
database when we actually only want 1. This is because Rails sees
`ApplicationRecord` and thinks it's a new connection, not the existing
`ActiveRecord::Base` connection because the
`connection_specification_name` is different.

This PR changes `ApplicationRecord` classes to consider themselves the
same as the "primary" connection.

Fixes #36382

When one database existed already, but not the other,
during setup of missing one, existing database was wiped out.