Deprecate and replace `set_state` method

`set_state` was directly setting the transaction state instance
variable. It's better to set the state via specific methods (`rollback!`
and `commit!` respectively.

While undocumented and untested, it's possible someone is using
`set_state` in their app or gem so I've added a deprecation notice to
it.

No where in the app do we use `nullify!` but I wanted to keep existing
behavior while replacing the method with a better pattern.

activerecord/CHANGELOG.md

+*   Deprecate `set_state` method in `TransactionState`
+
+    Deprecated the `set_state` method in favor of setting the state via specific methods. If you need to mark the
+    state of the transaction you can now use `rollback!`, `commit!` or `nullify!` instead of
+    `set_state(:rolledback)`, `set_state(:committed)`, or `set_state(nil)`.
+
+    *Eileen M. Uchitelle*, *Aaron Patterson*
+
 *   Deprecate delegating to `arel` in `Relation`.
 
     *Ryuta Kamizono*

activerecord/lib/active_record/connection_adapters/abstract/transaction.rb

 module ActiveRecord
   module ConnectionAdapters
     class TransactionState
-      VALID_STATES = Set.new([:committed, :rolledback, nil])
-
       def initialize(state = nil)
         @state = state
       end
@@ -24,10 +22,33 @@ def completed?
       end
 
       def set_state(state)
-        unless VALID_STATES.include?(state)
+        ActiveSupport::Deprecation.warn(<<-MSG.squish)
+          The set_state method is deprecated and will be removed in
+          Rails 5.2. Please use rollback! or commit! to set transaction
+          state directly.
+        MSG
+        case state
+        when :rolledback
+          rollback!
+        when :committed
+          commit!
+        when nil
+          nullify!
+        else
           raise ArgumentError, "Invalid transaction state: #{state}"
         end
-        @state = state
+      end
+
+      def rollback!
+        @state = :rolledback
+      end
+
+      def commit!
+        @state = :committed
+      end
+
+      def nullify!
+        @state = nil
       end
     end
 
@@ -57,7 +78,7 @@ def add_record(record)
       end
 
       def rollback
-        @state.set_state(:rolledback)
+        @state.rollback!
       end
 
       def rollback_records
@@ -72,7 +93,7 @@ def rollback_records
       end
 
       def commit
-        @state.set_state(:committed)
+        @state.commit!
       end
 
       def before_commit_records

activerecord/test/cases/transactions_test.rb

@@ -725,6 +725,44 @@ def test_transactions_state_from_commit
     assert transaction.state.committed?
   end
 
+  def test_set_state_method_is_deprecated
+    connection = Topic.connection
+    transaction = ActiveRecord::ConnectionAdapters::TransactionManager.new(connection).begin_transaction
+
+    transaction.commit
+
+    assert_deprecated do
+      transaction.state.set_state(:rolledback)
+    end
+  end
+
+  def test_mark_transaction_state_as_committed
+    connection = Topic.connection
+    transaction = ActiveRecord::ConnectionAdapters::TransactionManager.new(connection).begin_transaction
+
+    transaction.rollback
+
+    assert_equal :committed, transaction.state.commit!
+  end
+
+  def test_mark_transaction_state_as_rolledback
+    connection = Topic.connection
+    transaction = ActiveRecord::ConnectionAdapters::TransactionManager.new(connection).begin_transaction
+
+    transaction.commit
+
+    assert_equal :rolledback, transaction.state.rollback!
+  end
+
+  def test_mark_transaction_state_as_nil
+    connection = Topic.connection
+    transaction = ActiveRecord::ConnectionAdapters::TransactionManager.new(connection).begin_transaction
+
+    transaction.commit
+
+    assert_equal nil, transaction.state.nullify!
+  end
+
   def test_transaction_rollback_with_primarykeyless_tables
     connection = ActiveRecord::Base.connection
     connection.create_table(:transaction_without_primary_keys, force: true, id: false) do |t|