Co-Authored-By: NBob Whitelock <bob.whitelock1@gmail.com>
Co-Authored-By: NRafael França <rafael@franca.dev>
Co-Authored-By: NKapilSachdev <kapilsachdev03@gmail.com>

`connectes_to` should only be called on `ActiveRecord::Base` or abstract
classes. This is recommended in the documentation but until now was not
enforced by the code. It's unsafe to open too many connections to mysql
(and probably other databases), so it's safest to have 1 class for the
connection and subclass from that.
Co-authored-by: NJohn Crepezzi <john.crepezzi@gmail.com>

Active Record already supports adding `NOT VALID` foreign key constraints in PostgreSQL, but back in #31323 when check constraint support was initially added, the ability to add a check constraint and validate it separately was not included. This adds that functionality!

The handling for single database applications has always set a schema.rb
or structure.sql files for loading the database schema. When we first
implemented multiple database support we intended to keep this for the
original, default database. Afterall Rails _has_ to connect to something
on boot. In development only one connection is connected on boot since
we don't eager load the app.

Originally we had thought that all applications should be required to
add a `primary` entry in the database configurations file. However,
this hasn't worked in practice and we have some code now that does not
assume there's a primary. The schema dumping/loading code however,
still assumed there was a "primary" in the configurations file.

We want the "default" database in any application to use the original
files even when converted to a multiple database application as this
reduces the need to make changes when implementing this functionality on
an existing application.

The changes here update Rails to ensure that we treat either "primary"
or the first database configuration for an environment as "default".
If there is a "primary" that will be used as the default configuration.
If there is no primary the configuration that is first for an
environment will be used as the default. For schema dump/load this means
that the default configuration (primary or first) will use `schema.rb`
as the filename and other configurations will use
`[CONFIGURATION_NAME]_schema.rb`.

This should also help us finish the pull request to infer migrations
paths since now we can say the first configuration is the default. This
is a natural assumption for application developers.

Followup to #39536

This is an alternative of #29722.

Before Rails 6.1, storing demodulized class name is supported only for
STI type by `store_full_sti_class` class attribute.

Now `store_full_class_name` class attribute can handle both STI and
polymorphic types.

Closes #29722.

See also #29601, #32048, #32148.

Related to #35210.

We sometimes use `select` to limit unused columns for performance.

For example, `GET /posts/1` (post detail) usually use (almost) all
columns, but `GET /posts` (post list) does not always use all columns
(e.g. use `id` and `title` for the list view, but `body` is not used).

If an association is eager loaded, the limited `select` doesn't works as
expected, eager loading will load all columns on the model, plus also
load the `select` columns additionally. It works differently with
natural load and preload. It means that changing natural load or preload
to eager load (or vice versa) is unsafe.

This fixes eager loading that always load all columns (plus extra
`select` columns), to respect the `select` columns like as others.

```ruby
post = Post.select("UPPER(title) AS title").first
post.title # => "WELCOME TO THE WEBLOG"
post.body  # => ActiveModel::MissingAttributeError

# Rails 6.0 (ignore the `select` values)
post = Post.select("UPPER(title) AS title").eager_load(:comments).first
post.title # => "Welcome to the weblog"
post.body  # => "Such a lovely day"

# Rails 6.1 (respect the `select` values)
post = Post.select("UPPER(title) AS title").eager_load(:comments).first
post.title # => "WELCOME TO THE WEBLOG"
post.body  # => ActiveModel::MissingAttributeError
```

This reverts commit b8064500, reversing
changes made to 8714b359.

Reason: This is not approved from the core team yet...

```ruby
class Post < ActiveRecord::Base
  attribute :written_at, default: -> { Time.now.utc }
end

# Rails 6.0
Post.type_for_attribute(:written_at) # => #<Type::Value ... precision: nil, ...>

# Rails 6.1
Post.type_for_attribute(:written_at) # => #<Type::DateTime ... precision: 6, ...>
```

This is an alternative of #39797.

Context https://github.com/rails/rails/pull/39797#issuecomment-655191817.

If people intend to override the existing type on the attribute, usually
an overriding type is explicitly specified, so I agree that the current
behavior (drop existing type information if type is omitted) is
practically quite useless, and it is almost like a bug in that there is
no way to override just the default.

So I'd like to change the current behavior on existing attributes
without a deprecation as a fix.

Closes #39797.

See also https://github.com/kufu/activerecord-bitemporal/pull/57.

```ruby
class Book < ActiveRecord::Base
  enum status: [:proposed, :written, :published], _default: :published
end

Book.new.status # => "published"
```

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("create_at >=": time)` is better alternative than `where("create_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

`LegacyYamlAdapter` was added at afc124c3 (and 4e721702), to be able to
load the previous version formatted YAML.

Now, that has became quite useless, since we sometimes removed/added
dedicated type classes which are referred in the legacy format (e.g.
`MysqlDateTime`, `MysqlString`, `SQLite3Integer`), so it is unlikely
that is able to load it correctly any longer.

In Rails 4.2 we introduced mutation detection, to remove the need to
call `attribute_will_change!` after modifying a field. One side effect
of that change was that we needed to enforce that the
`_before_type_cast` form of a value is a different object than the post
type cast value, if the value is mutable. That contract is really only
relevant for strings, but it meant we needed to dup them.

In Rails 5 we added the `ImmutableString` type, to allow people to opt
out of this duping in places where the memory usage was causing
problems, and they don't need to mutate that field.

This takes that a step further, and adds a class-level setting to
specify whether strings should be frozen by default or not. The default
value of this setting is `false` (strings are mutable), and I do not
plan on changing that.

While I think that immutable strings by default would be a reasonable
default for new applications, I do not think that we would be able to
document the value of this setting in a place that people will be
looking when they can't figure out why some string is frozen.
Realistically, the number of applications where this setting is relevant
is fairly small, so I don't think it would make sense to ever enable it
by default.

These actually does not inplace mutate result. Use true `map` instead.

As described at #39328, `relation.merge` behaves inspired as Hash-like
merge for where clause. In other words, currently there is no official
way to intersect the result by both relation conditions (i.e. there is
no official way to maintain both relation conditions).

To resolve that issue, I'd like to support a way to intersect relations
as `relation.and`.

```ruby
david_and_mary = Author.where(id: [david, mary])
mary_and_bob   = Author.where(id: [mary, bob]) # => [bob]

david_and_mary.merge(mary_and_bob) # => [mary, bob]

david_and_mary.and(mary_and_bob) # => [mary]
david_and_mary.or(mary_and_bob)  # => [david, mary, bob]
```

Fixes #39232.

Related to #39250 and #39236.

The purpose of the change here is to unify inconsistent behavior on the
merging.

For now, mergee side condition is replaced by merger side condition only
if both arel nodes are Equality or In clauses.

In other words, if mergee side condition is not Equality or In clauses
(e.g. `between`, `or`, `gt`, `lt`, etc), those conditions will be kept
even on the same column.

This behavior is harder to predict unless people are familiar with the
merging behavior.

Originally I suppose this behavior is just an implementation issue
rather than an intended one, since `unscope` and `rewhere`, which were
introduced later than `merge`, works more consistently.

Since most of the conditions are usually Equality and In clauses, I
don't suppose most people have encountered this merging issue, but I'd
like to deprecate the inconsistent behavior and will completely unify
that to improve a future UX.

```ruby
# Rails 6.1 (IN clause is replaced by merger side equality condition)
Author.where(id: [david.id, mary.id]).merge(Author.where(id: bob)) # => [bob]

# Rails 6.1 (both conflict conditions exists, deprecated)
Author.where(id: david.id..mary.id).merge(Author.where(id: bob)) # => []

# Rails 6.1 with rewhere to migrate to Rails 6.2's behavior
Author.where(id: david.id..mary.id).merge(Author.where(id: bob), rewhere: true) # => [bob]

# Rails 6.2 (same behavior with IN clause, mergee side condition is consistently replaced)
Author.where(id: [david.id, mary.id]).merge(Author.where(id: bob)) # => [bob]
Author.where(id: david.id..mary.id).merge(Author.where(id: bob)) # => [bob]
```

Do not mark Postgresql MAC address and UUID attributes as changed when the assigned value only varies by case.

Fix Rubocop warnings.

Fix merge conflict error.

Fix merge conflict error.

Based on the docs which state unique_by option of insert_all can use the
index name if desired, one would expect the method to work normally and
use the `unique_by` option to determine duplicates.

However, there's an issue where the insert_all expects a Set instead
of the string representing the index expression it is given. This causes
an error. Returning the string expression instead of attempting to
format it works perfectly though.

Add `ActiveRecord::Base.strict_loading_by_default` and `ActiveRecord::Base.strict_loading_by_default=`.

This will allow to enable/disable strict_loading mode by default for a model.
The configuration's value is inheritable by subclasses, but they can override that value and
it will not impact parent class:

```ruby
class Developer < ApplicationRecord
  self.strict_loading_by_default = true

  has_many :projects
end

dev = Developer.first
dev.projects.first
\# => ActiveRecord::StrictLoadingViolationError Exception: Developer is marked as strict_loading and Project cannot be lazily loaded.
```

What is great about this feature that it could help users to nip N+1 queries in
the bud, especially for fresh applications, by setting
`ActiveRecord::Base.strict_loading_by_default = true` / `config.active_record.strict_loading_by_default = true`.
That is also a great way to prevent new N+1 queries in the existing applications
after all the N+1 queries are eliminated.
(See https://guides.rubyonrails.org/v6.0/active_record_querying.html#eager-loading-associations,
https://github.com/seejohnrun/prelude for details on how to fight against N+1 queries).

Related to https://github.com/rails/rails/pull/37400, https://github.com/rails/rails/pull/38541

Follow up to #27962.

#27962 only deprecated `quoted_id`, but still conservatively allowed
passing an Active Record object.

Since the quoting methods on a `connection` are low-level API and
querying API does not rely on that ability, so people should pass casted
value instead of an Active Record object if using the quoting methods
directly.

It doesn't matter if the transaction is rolled back or committed if it
wasn't written to, so we can avoid warning about a breaking change.

5 years ago, I made dumping full table options at #17569, especially to
dump `ENGINE=InnoDB ROW_FORMAT=DYNAMIC` to use utf8mb4 with large key
prefix.

In that time, omitting the default engine `ENGINE=InnoDB` was not useful
since `ROW_FORMAT=DYNAMIC` always remains as long as using utf8mb4 with
large key prefix.

But now, MySQL 5.7.9 has finally changed the default row format to
DYNAMIC, utf8mb4 with large key prefix can be used without dumping the
default engine and the row format explicitly.

So now is a good time to make the default engine is omitted.

Before:

```ruby
create_table "accounts", options: "ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci", force: :cascade do |t|
end
```

After:

```ruby
create_table "accounts", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
end
```

To entirely omit `:options` option to make schema agnostic, I've added
`:charset` and `:collation` table options to exclude `CHARSET` and
`COLLATE` from `:options`.

Fixes #26209.
Closes #29472.

See also #33608, #33853, and #34742.

We've learned that `merge` causes duplicated multiple values easily, so
if we missed to deduplicate the values, it will cause weird behavior
like #38052, #39171.

I've investigated the deduplication for the values, at least that had
existed since Rails 3.0.

https://github.com/rails/rails/commit/bed9179aa1496f6d28891cf515af0d7e515ebbab

Aggregations with group by multiple fields was introduced at Rails 3.1,
but we missed the deduplication for the aggregation result, unlike the
generated SQL.

https://github.com/rails/rails/commit/a5cdf0b9eb860c4370ae5fde231e1b61f71b6b65

While the investigation, I've found that `annotate` is also missed the
deduplication.

I don't suppose this weird behavior is intended for both.

So I'd like to deprecate the duplicated behavior in Rails 6.1, and will
be deduplicated all multiple values in Rails 6.2.

To migrate to Rails 6.2's behavior, use `uniq!(:group)` to deduplicate
group fields.

```ruby
accounts = Account.group(:firm_id)

# duplicated group fields, deprecated.
accounts.merge(accounts.where.not(credit_limit: nil)).sum(:credit_limit)
# => {
#   [1, 1] => 50,
#   [2, 2] => 60
# }

# use `uniq!(:group)` to deduplicate group fields.
accounts.merge(accounts.where.not(credit_limit: nil)).uniq!(:group).sum(:credit_limit)
# => {
#   1 => 50,
#   2 => 60
# }
```

Bump an Active Record instance's lock version after updating its counter
cache. This avoids raising an unnecessary ActiveRecord::StaleObjectError
upon subsequent transactions by maintaining parity with the
corresponding database record's lock_version column.

Related #39236.

`relation.merge` method sometimes replaces mergee side condition, but
sometimes maintain both conditions unless `relation.rewhere` is used.

It is very hard to predict merging result whether mergee side condition
will be replaced or not.

One existing way is to use `relation.rewhere` for merger side relation,
but it is also hard to predict a relation will be used for `merge` in
advance, except one-time relation for `merge`.

To address that issue, I propose to support merging option `:rewhere`,
to allow mergee side condition to be replaced exactly.

That option will allow non-`rewhere` relation behaves as `rewhere`d
relation.

```ruby
david_and_mary = Author.where(id: david.id..mary.id)

# both conflict conditions exists
david_and_mary.merge(Author.where(id: bob)) # => []

# mergee side condition is replaced by rewhere
david_and_mary.merge(Author.rewhere(id: bob)) # => [bob]

# mergee side condition is replaced by rewhere option
david_and_mary.merge(Author.where(id: bob), rewhere: true) # => [bob]
```

Add support for finding records based on signed ids, which are tamper-proof, verified ids that can be set to expire and scoped with a purpose. This is particularly useful for things like password reset or email verification, where you want the bearer of the signed id to be able to interact with the underlying record, but usually only within a certain time period.

This is the opposite direction of #39039.

#39111 fixes `minimum` and `maximum` on date columns with type casting
by column type on the database. But column type has no information for
time zone aware attributes, it means that attribute type should always
be precedence over column type. I've realized that fact in the related
issue report #39248.

I've reverted the expectation of #39039, to make time zone aware
attributes works.

Since MySQL 8.0.12, MySQL supports `ALGORITHM=INSTANT` DDL option.

https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html#online-ddl-index-operations

I've found the bug when I'm refactoring index creation code in #39203.

It is possible to unscope only the column in the specified table.

```ruby
posts = Post.joins(:comments).group(:"posts.hidden")
posts = posts.where("posts.hidden": false, "comments.hidden": false)

posts.count
# => { false => 10 }

# unscope both hidden columns
posts.unscope(where: :hidden).count
# => { false => 11, true => 1 }

# unscope only comments.hidden column
posts.unscope(where: :"comments.hidden").count
# => { false => 11 }
```
Co-authored-by: NSlava Korolev <korolvs@gmail.com>

```ruby
steve = Person.find_by(name: "Steve")
david = Author.find_by(name: "David")

relation = Essay.where(writer: steve)

# Before
relation.rewhere(writer: david).to_a # => []

# After
relation.rewhere(writer: david).to_a # => [david]
```

For now `rewhere` only works for truly column names, doesn't work for
alias attributes, nested conditions, associations.

To fix that, need to build new where clause first, and then get
attribute names from new where clause.

before
```
p Knot.create
=> #<Knot id: 1, created_at: "2016-05-05 01:29:47">
```

after
```
p Knot.create
=> #<Knot id: 1, created_at: "2016-05-05 01:29:47.116928000">
```