Follow-up c280ae72.

This PR adds support for `if_exists` on `remove_column` and
`if_not_exists` on `add_column` to support silently ignoring migrations
if the remove tries to remove a non-existent column or an add tries to
add an already existing column.

We (GitHub) have custom monkey-patched support for these features and
would like to upstream this behavior.

This matches the same behavior that is supported for `create_table` and
`drop_table`. The behavior for sqlite is different from mysql/postgres
and sqlite for remove column and that is reflected in the tests.

Follow-up of 1ee4a881.

This PR moves advisory lock to it's own connection instead of
`ActiveRecord::Base` to fix #37748. As a note the issue is present on
both mysql and postgres. We don't see it on sqlite3 because sqlite3
doesn't support advisory locks.

The underlying problem only appears if:

1) the app is using multiple databases, and therefore establishing a new
connetion in the abstract models
2) the app has a migration that loads a model (ex `Post.update_all`)
which causes that new connection to get established.

This is because when Rails runs migrations the default connections are
established, the lock is taken out on the `ActiveRecord::Base`
connection. When the migration that calls a model is loaded, a new
connection will be established and the lock will automatically be
released.

When Rails goes to release the lock in the ensure block it will find
that the connection has been closed. Even if the connection wasn't
closed the lock would no longer exist on that connection.

We originally considered checking if the connection was active, but
ultimately that would hide that the advisory locks weren't working
correctly because there'd be no lock to release.

We also considered making the lock more granular - that it only blocked
on each migration individually instead of all the migrations for that
connection. This might be the right move going forward, but right now
multi-db migrations that load models are very broken in Rails 6.0 and
master.

John and I don't love this fix, it requires a bit too much knowledge of
internals and how Rails picks up connections. However, it does fix the
issue, makes the lock more global, and makes the lock more resilient to
changing connections.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

This updates the database tasks for dumping the Active Record schema cache as
well as clearing the schema cache file, allowing the path to be defined in the
database configuration YAML file.

As before, the value can also be defined in an ENV variable, though this would
not work for a multi-db application. If the value is specified neither in the
DB config, nor in the ENV, then the path will continue to be derived from the
DB config spec_name.

Note that in order to make this change cleaner I also moved a bit of logic
out of a rake task and into the DatabaseTasks class, for symmetry.

We have two rake tasks for the schema cache:

    $ rake db:schema:cache:dump
    $ rake db:schema:cache:clear

The cache:dump task was implemented in DatabaseTasks, but the
cache:clear one was not.

I also added some tests for the behavior that I was changing, since some of
the code paths weren't tested.

Calling `#remove_connection` on the handler is deprecated in favor of
`#remove_connection_pool`. This change was made to support changing the
return value from a hash to a `DatabaseConfig` object.
`#remove_connection` will be removed in 6.2.

NOTE: `#remove_connection` on `ActiveRecord::Base` will also now return
a `DatabaseConfig` object. We didn't use a deprecation here since
it's not documented that this method used to return a `Hash`.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

Database configurations are now objects almost everywhere, so we don't
need to fake access to a hash with `#default_hash` or it's alias `#[]`.
Applications should `configs_for` and pass `env_name` and `spec_name` to
get the database config object. If you're looking for the default for
the test environment you can pass `configs_for(env_name: "test", spec_name:
"primary")`. Change test to developement to get the dev config, etc.

`#default_hash` and `#[]` will be removed in 6.2.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

Since test fixtures share connections (due to transactional tests) we
end up overwriting the reading configuration so Rails doesn't recognize
it as a replica connection.

This change ensures that if we're using the `reading` role that
connections will always have prevent writes turned on.

If you need a replica connection that does not block writes, you should
use a different role name other than `:reading`.

The db selector test and connection handlers test have been updated to
test for these changes. In the db selector test we don't always have a
writing handler so I updated test fixtures to return if that's nil.

Lastly one test needed to be updated to use a different handler name due
to it needing to write to successfully test what it needs to test.

Fixes #37765

As multiple databases have evolved it's becoming more and more
confusing that we have a `connection_specification_name` that defaults
to "primary" and a `spec_name` on the database objects that defaults to
"primary" (my bad).

Even more confusing is that we use the class name for all
non-ActiveRecord::Base abstract classes that establish connections. For
example connections established on `class MyOtherDatabaseModel <
ApplicationRecord` will use `"MyOtherDatabaseModel"` as it's connection
specification name while `ActiveRecord::Base` uses `"primary"`.

This PR deprecates the use of the name `"primary"` as the
`connection_specification_name` for `ActiveRecord::Base` in favor of
using `"ActiveRecord::Base"`.

In this PR the following is true:

* If `handler.establish_connection(:primary)` is called, `"primary"`
will not throw a deprecation warning and can still be used for the
`connection_specification_name`. This also fixes a bug where using this
method to establish a connection could accidentally overwrite the actual
`ActiveRecord::Base` connection IF that connection was not using a
configuration named `:primary`.
* Calling `handler.retrieve_connection "primary"` when
`handler.establish_connection :primary` has never been called will
return the connection for `ActiveRecord::Base` and throw a deprecation
warning.
* Calling `handler.remove_connection "primary"` when
`handler.establish_connection :primary` has never been called will
remove the connection for `ActiveRecord::Base` and throw a deprecation
warning.

See #38179 for details on more motivations for this change.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

Add Active Record numericality validator with support for casting
floats using a database columns' precision value.

Add ActiveRecord::Relation#cache_key_with_version. This method will be
used by ActionController::ConditionalGet to ensure that when collection
cache versioning is enabled, requests using ConditionalGet don't return
the same ETag header after a collection is modified.

Prior to the introduction of collection cache versioning in
4f2ac80d, all collection cache keys
included a version. However, with cache versioning enabled, collection
cache keys remain constant. In turn, ETag headers remain constant,
rendering them ineffective.

This commit takes the cache_key_with_version method used for individual
Active Record objects (from aa8749eb),
and adds it to collections.

Fixes https://github.com/rails/rails/issues/28827.

The steps to reproduce are as follows:

git clone git@github.com:bbuchalter/rails-issue-28827.git
cd rails-issue-28827
bundle install
bin/rails db:create

Observe that we create two databases when invoking db:create: development and test. Now observe what happens when we invoke our drop command while using DATABASE_URL.

DATABASE_URL=sqlite3://$(pwd)/db/database_url.sqlite3 bin/rails db:create

As expected, the development environment now uses the DATABASE_URL. What is unexpected is that the test environment does not.

It's unclear what the expected behavior should be in this case, but the cause of it is this: https://github.com/rails/rails/blob/9f2c74eda07ea5a9e4e624d5575a717714088dbf/activerecord/lib/active_record/tasks/database_tasks.rb#L494

Because of each_local_configuration, there seems to be no way invoke these database rake on only the development environment to ensure DATABASE_URL is respected.

The smallest scope of change I can think to make would be to conditionalize this behavior so it does not get applied when DATABASE_URL is present.

`name` is used by Rails to find the configuration by connection
specification name, but database adapters don't need to use `name` in
order to establish a connection. This is part of our work to separate
what the database needs to connect (the configuration hash) and the
what Rails needs to find connections (everything else).
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

- I'm hoping to get this change accepted even though this flag was
  introduced years ago in 6576f735

  My use case is the following:

  We were never enforcing belongs to association and we have a lot
  of models that implicitely declare an association as optional.
  We are now changing all our models to make associations required
  by default.
  Since we have a lot of models (more than 1000), I'd like to
  progressively enable them to use the `belongs_to_required_by_default`
  flag.

  The problem is that this flag is a mattr_accessor and doesn't to be
  set per model. We basically need to modify all our models (which
  could take years) before being able to modify the global flag.

  I'd like to change this flag to a class_attribute to solve the
  issue.

The `connection_config` method returns a `Hash`, but since we're moving
toward a place where we're using `DatabaseConfiguration::DatabaseConfig`
objects everywhere, we're introducing a new method here to replace it
called `connection_db_config`.
Co-authored-by: Neileencodes <eileencodes@gmail.com>

A querystring value should be allowed to include an equal sign `=`.

This is necessary to support passing `options` for a PostgresSQL connection.

```

development:
  url: postgresql://localhost/railsdevapp_development?options=-cmysetting.debug=on
```

Before this PR, attempting to start the rails process with that configuration would result in an error:

```
> bundle exec rails console
Traceback (most recent call last):
	49: from bin/rails:4:in `<main>'
	48: from bin/rails:4:in `require'
...
	 1: from /rails/activerecord/lib/active_record/database_configurations/connection_url_resolver.rb:58:in `query_hash'
/rails/activerecord/lib/active_record/database_configurations/connection_url_resolver.rb:58:in `[]': invalid number of elements (3 for 1..2) (ArgumentError)
```

After this PR, rails can properly parse the configuration:

```
> bundle exec rails console
Loading development environment (Rails 6.1.0.alpha)
2.6.5 :001 > ActiveRecord::Base.connection.select_all("show mysetting.debug").to_a
   (0.4ms)  show mysetting.debug
 => [{"mysetting.debug"=>"on"}]
```

Applying `includes` and `joins` to a relation that selected additional
database fields would cause those additional fields not to be included
in the results even though they were queried from the database:

    posts = Post.select('1 as other').includes(:tbl).joins(:tbl)

    posts.to_sql.include?('1 as other')       #=> true
    posts.first.attributes.include?('other')  #=> false

This commit includes these additionally selected fields in the
instantiated results.

The `database` kwarg in `connected_to` has resulted in a lot of bug
reports that are trying to use it for sharding when that's not the
intent of the key. After considering where the database kwarg is used in
tests and thinking about usecases for it, we've determined it should be
removed.

There are plans to add sharding support and in the mean time the
database kwarg isn't the right solution for that. Applications that need
to create new connections can use establish_connection or connects_to.
Since the database key causes new connections to be established on every
call, that causes bugs if connected_to with a database kwarg is used
during a request or for any connection that's not a one-off.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

We have these two objects, `ConnectionAdapters::Resolver` and
`DatabaseConfiguratons` that implement a lot of the same logic. One of
them is used for configurations defined in `config/database.yml` and the
other is used when passing raw configurations `String` or `Hash` objects
into methods like `establish_connection`.

Over time these two have diverged a bit. In the interest of less code
complexity, and more consistency for users this commit brings them back
together.

* Remove `Resolver` altogether and replace its primary method with
  `DatabaseConfigurations#resolve`.

* Move `resolve_pool_config` over to the `ConnectionPool` alongside the code
  that uses it.

Forgot to add a changelog when this was merged in #37266.

We deprecated `to_h` in https://github.com/rails/rails/pull/37231, and
this commit adds the appropriate `CHANGELOG` entry!

Previously in some places we used symbol keys, and in some places we used
string keys. That made it pretty confusing to figure out in a particular
place what type of configuration object you were working with.

Now internally, all configuration hashes are keyed by symbols and
converted to such on the way in.

A few exceptions:

- `DatabaseConfigurations#to_h` still returns strings for backward compatibility
- Same for `legacy_hash`
- `default_hash` previously could return strings, but the associated
  comment mentions it returns symbol-key `Hash` and now it always does

Because this is a change in behavior, a few method renames have happened:

- `DatabaseConfig#config` is now `DatabaseConfig#configuration_hash` and returns a symbol-key `Hash`
- `ConnectionSpecification#config` is now `ConnectionSpecification#underlying_configuration_hash` and returns the `Hash` of the underlying `DatabaseConfig`
- `DatabaseConfig#config` was added back, returns `String`-keys for backward compatibility, and is deprecated in favor of the new `configuration_hash`
Co-authored-by: Neileencodes <eileencodes@gmail.com>

Before this patch, column names could only be passed as a positional
argument when no other options were supplied:

    remove_index :reports, :report_id

Passing column names positionally along with other options would fail:

    remove_index :reports, :report_id, unique: true
    # => ArgumentError: wrong number of arguments (given 3, expected 1..2)

In 3809c80c, adding an index with a
name that's already in use was changed from an error to a warning, to
allow other statements in the same migration to complete successfully.

In 55d0d57b this decision was reversed,
but instead of allowing the statement to execute and raise an adapter-
specific error as it did before, an `ArgumentError` was raised instead.

This interferes with a legitimate use case: on MySQL, it's possible to
drop an index and add another one with the same name in a single `ALTER`
statement. Right now an `ArgumentError` is raised when trying to do so,
even though the resulting statement would execute successfully.

There's no corresponding `ArgumentError` raised when attempting to add a
duplicate column, so I think we can safely remove the check and allow
the adapter to raise its own error about duplicate indexes again.

The `InsertAll` class currently calls `exec_query`, which doesn't give
the query cache enough information to know that it needs to be cleared.

By adding an `exec_insert_all` method that calls `exec_query` internally
we can configure the query cache to clear when that method is called.

If a user is using the middleware for swapping database connections and
manually calling `connected_to` in a controller/model/etc without
calling `while_preventing_writes(false)` there is potential for a race
condition where writes will be blocked.

While the user could _just_ call `while_preventing_writes` in the same
place they call `connected_to` this would mean that all cases need to
call two methods.

This PR changes `connected_to` to call `while_preventing_writes`
directly. By default we'll assume you don't want to prevent writes, but
if called with `connected_to(role: :writing, prevent_writes: true)` or
from the middleware (which calls `connected_to` this way) the writes
will be blocked.

For replicas, apps should use readonly users to enforce not writing
rather than `while_preventing_writes` directly.

Should fix the remaining issues in
https://github.com/rails/rails/issues/36830

in the edge case when the query is terminated by MySQL server during filesort.
See https://bugs.mysql.com/bug.php?id=96537 for more details.