Didn't like the complicated stuff that happened on credentials:edit. It
would append to .gitattributes multiple times. Though I see why it was
written that way.

I'm cutting off for now, but since this new flow would require each developer
to run --enable perhaps this should really be:

1. Developer enrolls Rails app by running `credentials:diff --enable`
2. credentials:edit checks .gitattributes for `diff=rails_credentials` and
   if the current file is covered by that.
3. If so, set up the "rails_credentials" driver automatically.

This reverts commit 98d0f7eb.

It's used everywhere, clean and mature enough

- @sinsoku had the idea and started implementing it few months ago
  but sadly didn't finish it.
  This PR is taking over his work.

  The credentials feature has changed a lot since @sinsoku opened hi
  PR, it was easier to just restart from scratch instead of checking
  out his branch.
  Sinsoku will get all the credit he deserves for this idea :)

  TL;DR on that that feature is to make the `git diff` or `git log`
  of encrypted files to be readable.

  The previous implementation was only setting up the git required
  configuration for the first time Rails was bootstraped, so I decided
  to instead provide the user a choice to opt-in for readable diff
  credential whenever a user types the `bin/rails credentials:edit`
  command.
  The question won't be asked in the future the user has already
  answered or if the user already opted in.
Co-authored-by: NTakumi Shotoku <insoku.listy@gmail.com>

When one database existed already, but not the other,
during setup of missing one, existing database was wiped out.

This is kind of hard to explain but if you have a database config with
another level like this:

```
development:
  primary:
    database: "my db"
    variables:
      statement_timeout: 1000
```

the database configurations code would chooke on the `variables` level
because it didn't know what to do with it.

We'd see the following error:

```
lib/active_record/database_configurations.rb:72:in
`block in find_db_config': undefined method `env_name' for [nil]:Array
(NoMethodError)
```

The problem here is that Rails does correctly identify this as not a
real configuration but returns `[nil]` along with the others. We need to
make sure to flatten the array and remove all the `nil`'s before
returning the `configurations` objects.

Fixes #35646

If the secret_key_base is nil in dev or test generate a key from random
bytes and store it in a tmp file. This prevents the app developers from
having to share / checkin the secret key for dev / test but also
maintains a key between app restarts in dev/test.

[CVE-2019-5420]
Co-Authored-By: Neileencodes <eileencodes@gmail.com>
Co-Authored-By: NJohn Hawthorn <john@hawthorn.email>

In CI, these are unnecessary because done by preprocessing.
To test locally, you need to do it manually. If assets not exist, it will
fail when a run specific test, so force to build those only when necessary.

Zeitwerk is a strong dependency, planned to replace AS::Dependencies. A
line in the generated Gemfile does not convey this as much.

Some tests are running yarn install during the test.
The directory used for isolation test is not subject to yarn workspace,
and it occurs because the required package is not installed.
In order to avoid this, I fixed all necessary packages to be installed
before run test and use symlink to `node_modules`.

This is a bit complicated, as `yarn install` needs to be run in a specific
directory before running the test.
However, running `yarn install` every time run the test is expensive
when testing locally and should be avoided.

This reverts commit 38f9e41f, reversing
changes made to 5e493c3b.

Reason: The ajv@6.9.1 was released that fixes issue.

Because ajv@6.9.0 does not work with webpack.
Ref: https://github.com/epoberezkin/ajv/issues/941

To avoid running require individually.

Add `rails db:system:change` command for changing databases.

```
bin/rails db:system:change --to=postgresql
   force  config/database.yml
    gsub  Gemfile
```

The change command copies a template `config/database.yml` with
the target database adapter into your app, and replaces your database
gem with the target database gem.

Rails 6 requires Ruby 2.5, which introduces `FrozenError`
https://docs.ruby-lang.org/en/2.5.0/NEWS.html

Related to #31520

The ActionDispatch::HostAuthorization is a new middleware that prevent
against DNS rebinding and other Host header attacks. By default it is
included only in the development environment with the following
configuration:

    Rails.application.config.hosts = [
      IPAddr.new("0.0.0.0/0"), # All IPv4 addresses.
      IPAddr.new("::/0"),      # All IPv6 addresses.
      "localhost"              # The localhost reserved domain.
    ]

In other environments, `Rails.application.config.hosts` is empty and no
Host header checks will be done. If you want to guard against header
attacks on production, you have to manually permit the allowed hosts
with:

    Rails.application.config.hosts << "product.com"

The host of a request is checked against the hosts entries with the case
operator (#===), which lets hosts support entries of type RegExp,
Proc and IPAddr to name a few. Here is an example with a regexp.

    # Allow requests from subdomains like `www.product.com` and
    # `beta1.product.com`.
    Rails.application.config.hosts << /.*\.product\.com/

A special case is supported that allows you to permit all sub-domains:

    # Allow requests from subdomains like `www.product.com` and
    # `beta1.product.com`.
    Rails.application.config.hosts << ".product.com"

* Use Webpacker by default on new apps

* Stop including coffee-rails by default

* Drop using a js_compressor by default

* Drop extra test for coffeescript inclusion by default

* Stick with skip_javascript to signify skipping webpack

* Don't install a JS runtime by default any more

* app/javascript will be the new default directory for JS

* Make it clear that this is just for configuring the default Webpack framework setup now

* Start using the Webpack tag in the default layout

* Irrelevant test

* jQuery is long gone

* Stop having asset pipeline compile default application.js

* Add rails-ujs by default to the Webpack setup

* Add Active Storage JavaScript to application.js pack by default

* Consistent quoting

* Add Turbolinks to default pack

* Add Action Cable to default pack

Need some work on how to set the global consumer that channels will
work with. @javan?

* Require all channels by default and use a separate consumer stub

* Channel generator now targets Webpack style

* Update task docs to match new generator style

* Use uniform import style

* Drop the JS assets generator

It was barely helpful as it was. It’s no longer helpful in a Webpacked
world. Sayonara!

* Add app/javascript to the stats directories

* Simpler import style

Which match the other imports.

* Address test failures from dropping JS compilation (and compression)

* webpacker-default: Modify `AssetsGeneratorTest`

Before:

```
$ bin/test test/generators/assets_generator_test.rb
Run options: --seed 46201

F

Failure:
AssetsGeneratorTest#test_assets [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/assets_generator_test.rb:12]:
Expected file "app/assets/javascripts/posts.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/assets_generator_test.rb:10

.

Finished in 0.031343s, 63.8101 runs/s, 95.7152 assertions/s.
2 runs, 3 assertions, 1 failures, 0 errors, 0 skips
```

After:

```
$ bin/test test/generators/assets_generator_test.rb
Run options: --seed 43571

..

Finished in 0.030370s, 65.8545 runs/s, 65.8545 assertions/s.
2 runs, 2 assertions, 0 failures, 0 errors, 0 skips
```

* webpacker-default: Modify `ChannelGeneratorTest`

Before:

```
$ bin/test test/generators/channel_generator_test.rb
Run options: --seed 8986

.F

Failure:
ChannelGeneratorTest#test_channel_with_multiple_actions_is_created [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:43]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:34

.F

Failure:
ChannelGeneratorTest#test_channel_is_created [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:29]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:22

E

Error:
ChannelGeneratorTest#test_cable_js_is_created_if_not_present_already:
Errno::ENOENT: No such file or directory @ apply2files - /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/fixtures/tmp/app/assets/javascripts/cable.js

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:60

F

Failure:
ChannelGeneratorTest#test_channel_suffix_is_not_duplicated [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:87]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:80

F

Failure:
ChannelGeneratorTest#test_channel_on_revoke [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:77]:
Expected file "app/assets/javascripts/cable.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:68

Finished in 0.064384s, 108.7227 runs/s, 481.4861 assertions/s.
7 runs, 31 assertions, 4 failures, 1 errors, 0 skips
```

After:

```
$ bin/test test/generators/channel_generator_test.rb
Run options: --seed 44857

.......

Finished in 0.060243s, 116.1961 runs/s, 697.1764 assertions/s.
7 runs, 42 assertions, 0 failures, 0 errors, 0 skips
```

* Fix shared generator tests.

* webpacker-default: Modify `ControllerGeneratorTest`

The JS assets generator was dropped. ref. https://github.com/rails/rails/commit/46215b179483d3e4d264555f5a4952f43eb8142a

* Revert "Simpler import style". It's currently failing with an error of "TypeError: undefined is not an object (evaluating '__WEBPACK_IMPORTED_MODULE_2_activestorage___default.a.start')". Waiting for @javan to have a look.

This reverts commit 5d3ebb71059f635d3756cbda4ab9752027e09256.

* require webpacker in test app

* Add webpacker without making the build hang/timeout. (#33640)

* use yarn workspaces to allow for installing unreleased packages and only generate js/bootsnap when required

* no longer need to have webpacker in env templates as webpacker moved this config to yml file

* Fix rubocop violation

* Got the test passing for the running scaffold

* update expected lines of code

* update middleware tests to account for webpacker

* disable js in plugins be default to get the tests passing (#34009)

* clear codeclimate report issues

* Anything newer than currently released is good

* Use Webpacker development version during development of Rails

* Edge should get development webpacker as well

* Add changelog entry for Webpacker change

In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.

```ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Benchmark.ips do |x|
  x.report('+@') { +"" }
  x.report('dup') { "".dup }
  x.compare!
end
```

```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
                  +@   282.289k i/100ms
                 dup   187.638k i/100ms
Calculating -------------------------------------
                  +@      6.775M (± 3.6%) i/s -     33.875M in   5.006253s
                 dup      3.320M (± 2.2%) i/s -     16.700M in   5.032125s

Comparison:
                  +@:  6775299.3 i/s
                 dup:  3320400.7 i/s - 2.04x  slower

```

These were added for avoiding warnings and for testing in
e4c529ea and 6ea7065a.

Now the default is `:random`, and since the tests added with it are
removed. That config is unnecessary.

Changes the `configs_for` method from using traditional arguments to
using kwargs. This is so I can add the `include_replicas` kwarg without
having to always include `env_name` and `spec_name` in the method call.

`include_replicas` defaults to false because everywhere internally in
Rails we don't want replicas. `configs_for` is for iterating over
configurations to create / run rake tasks, so we really don't ever need
replicas in that case.

Include `ActiveSupport::Testing::MethodCallAssertions` in `railties/test/isolation/abstract_unit.rb`

Related to #33102

I realized I wasn't really testing some of the new rake tasks added so I
built out this new test that uses a multi-db database.yml and allows us
to run create/drop/migrate/schema:dump/schema:load and those that are
namespaced like create:animals. This will make our testing more robust
so we can catch problems quicker and set a good place to add future
tests as these features evolve.

Provides both a forked process and threaded parallelization options. To
use add `parallelize` to your test suite.

Takes a `workers` argument that controls how many times the process
is forked. For each process a new database will be created suffixed
with the worker number; test-database-0 and test-database-1
respectively.

If `ENV["PARALLEL_WORKERS"]` is set the workers argument will be ignored
and the environment variable will be used instead. This is useful for CI
environments, or other environments where you may need more workers than
you do for local testing.

If the number of workers is set to `1` or fewer, the tests will not be
parallelized.

The default parallelization method is to fork processes. If you'd like to
use threads instead you can pass `with: :threads` to the `parallelize`
method. Note the threaded parallelization does not create multiple
database and will not work with system tests at this time.

parallelize(workers: 2, with: :threads)

The threaded parallelization uses Minitest's parallel exector directly.
The processes paralleliztion uses a Ruby Drb server.

For parallelization via threads a setup hook and cleanup hook are
provided.

```
class ActiveSupport::TestCase
  parallelize_setup do |worker|
    # setup databases
  end

  parallelize_teardown do |worker|
    # cleanup database
  end

  parallelize(workers: 2)
end
```

[Eileen M. Uchitelle, Aaron Patterson]

* `assert_header` and `assert_body` were unused since 6f6a589d.
* `assert_success` and `assert_missing` were unused since added.

This pull request handles `FrozenError` introduced by Ruby 2.5.
Refer https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61131

Since `FrozenError` is a subclass of `RuntimeError` minitest used by master
branch can handle it, though it would be better to handle `FrozenError`
explicitly if possible.

`FrozenError` does not exist in Ruby 2.4 or lower, `frozen_error_class`
handles which exception is expected to be raised.

This pull request is intended to be merged to master,
then backported to `5-1-stable` to address #31508

I frequently run tests with `ruby`, not with a runner like `rake` or
`rails`.  When running the test with just `ruby` the `RAILS_ENV`
environment variable did not get set to "test", and this would cause the
tests to fail (and even mutate the development database!)

This commit adds integration tests for running tests with just `ruby`
and ensures the environment gets defaulted to "test".  I also added a
test to ensure that passing an environment to `-e` actually works (and
fixed that case too).

An interesting / annoying thing is that Minitest picks up it's plugins
by asking RubyGems for a list of files:

  https://github.com/seattlerb/minitest/blob/ca6a71ca901016db09a5ad466b4adea4b52a504a/lib/minitest.rb#L92-L100

This means that RubyGems needs to somehow know about the file before it
can return it to Minitest.  Since we are not packaging Rails as a Gem
before running the integration tests on it (duh, why would you do
that?), RubyGems doesn't know about the file, so it can't tell Minitest,
so Minitest doesn't automatically require it.  This means I had to
manually require and insert the plugin in our integration test.  I've
left comments about that in the test as well.

Ugh.

This is necessary in order to make the processing dependent on
`Migrator.current_version` work even without database.

Context: https://github.com/rails/rails/pull/31135#issuecomment-348404326

Omit `rails activestorage:install` for jdbcmysql, jdbc and shebang tests

AppGeneratorTest#test_config_jdbcmysql_database

  rails aborted!
  LoadError: Could not load 'active_record/connection_adapters/mysql_adapter'.
  Make sure that the adapter in config/database.yml is valid.
  If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add
  the necessary adapter gem to the Gemfile.
  (compressed)
  bin/rails:4:in `<main>'
  Tasks: TOP => activestorage:install => environment
  (See full trace by running task with --trace)

AppGeneratorTest#test_config_jdbc_database

  rails aborted!
  LoadError: Could not load 'active_record/connection_adapters/jdbc_adapter'.
  Make sure that the adapter in config/database.yml is valid.
  If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add
  the necessary adapter gem to the Gemfile.
  (compressed)
  bin/rails:4:in `<main>'
  Tasks: TOP => activestorage:install => environment
  (See full trace by running task with --trace)

AppGeneratorTest#test_shebang_is_added_to_rails_file

  /home/ubuntu/.rbenv/versions/2.4.1/bin/ruby: no Ruby script found in input (LoadError)

Prevent PendingMigrationError in tests

 * Run `bin/rails db:migrate RAILS_ENV=test` in test_cases before start tests to prevent PendingMigrationError
 * FileUtils.rm_r("db/migrate")
 * --skip-active-storage

Fix failed tests in `railties/test/railties/engine_test.rb`

Related to #30111

Imporve `SharedGeneratorTests#test_default_frameworks_are_required_when_others_are_removed`

 - Explicitly skip active_storage
 - Ensure that skipped frameworks are commented
 - Ensure that default frameworks are not commented

Fix error `Errno::ENOSPC: No space left on device - sendfile`

Since `rails new` runs `rails active_storage:install`
that boots an app.

Since adding Bootsnap 0312a5c6
during booting an app, it creates the cache:

   264K    tmp/cache/bootsnap-load-path-cache
   27M     tmp/cache/bootsnap-compile-cache

* teardown_app must remove app

Ref: https://github.com/rails/rails/blob/3be123ba26cad461a80d7d680819e71c1388a241/railties/test/isolation/abstract_unit.rb#L243

* WIP: Add credentials using a generic EncryptedConfiguration class

This is sketch code so far.

* Flesh out EncryptedConfiguration and test it

* Better name

* Add command and generator for credentials

* Use the Pathnames

* Extract EncryptedFile from EncryptedConfiguration and add serializers

* Test EncryptedFile

* Extract serializer validation

* Stress the point about losing comments

* Allow encrypted configuration to be read without parsing for display

* Use credentials by default and base them on the master key

* Derive secret_key_base in test/dev, source it from credentials in other envs

And document the usage.

* Document the new credentials setup

* Stop generating the secrets.yml file now that we have credentials

* Document what we should have instead

Still need to make it happen, tho.

* [ci skip] Keep wording to `key base`; prefer defaults.

Usually we say we change defaults, not "spec" out a release.

Can't use backticks in our sdoc generated documentation either.

* Abstract away OpenSSL; prefer MessageEncryptor.

* Spare needless new when raising.

* Encrypted file test shouldn't depend on subclass.

* [ci skip] Some woordings.

* Ditch serializer future coding.

* I said flip it. Flip it good.

* [ci skip] Move require_master_key to the real production.rb.

* Add require_master_key to abort the boot process.

In case the master key is required in a certain environment
we should inspect that the key is there and abort if it isn't.

* Print missing key message and exit immediately.

Spares us a lengthy backtrace and prevents further execution.

I've verified the behavior in a test app, but couldn't figure the
test out as loading the app just exits immediately with:

```
/Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `load': marshal data too short (ArgumentError)
	from /Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `run'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest.rb:830:in `run_one_method'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest/parallel.rb:32:in `block (2 levels) in start'
```

It's likely we need to capture and prevent the exit somehow.
Kernel.stub(:exit) didn't work. Leaving it for tomorrow.

* Fix require_master_key config test.

Loading the app would trigger the `exit 1` per require_master_key's
semantics, which then aborted the test.

Fork and wait for the child process to finish, then inspect the
exit status.

Also check we aborted because of a missing master key, so something
else didn't just abort the boot.

Much <3 to @tenderlove for the tip.

* Support reading/writing configs via methods.

* Skip needless deep symbolizing.

* Remove save; test config reader elsewhere.

* Move secret_key_base check to when we're reading it.

Otherwise we'll abort too soon since we don't assign the secret_key_base
to secrets anymore.

* Add missing string literal comments; require unneeded yaml require.

* ya ya ya, rubocop.

* Add master_key/credentials after bundle.

Then we can reuse the existing message on `rails new bc4`.

It'll look like:

```
Using web-console 3.5.1 from https://github.com/rails/web-console.git (at master@ce985eb)
Using rails 5.2.0.alpha from source at `/Users/kasperhansen/Documents/code/rails`
Using sass-rails 5.0.6
Bundle complete! 16 Gemfile dependencies, 72 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Adding config/master.key to store the master encryption key: 97070158c44b4675b876373a6bc9d5a0

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access anything encrypted with it.

      create  config/master.key
```

And that'll be executed even if `--skip-bundle` was passed.

* Ensure test app has secret_key_base.

* Assign secret_key_base to app or omit.

* Merge noise

* Split options for dynamic delegation into its own method and use deep symbols to make it work

* Update error to point to credentials instead

* Appease Rubocop

* Validate secret_key_base when reading it.

Instead of relying on the validation in key_generator move that into
secret_key_base itself.

* Fix generator and secrets test.

Manually add config.read_encrypted_secrets since it's not there by default
anymore.

Move mentions of config/secrets.yml to config/credentials.yml.enc.

* Remove files I have no idea how they got here.

* [ci skip] swap secrets for credentials.

* [ci skip] And now, changelogs are coming.

Since ff30db13, warning is not shown.

While this avoids shell argument parsing, we still pass through
everything in our stack.