1. 18 5月, 2020 2 次提交
  2. 15 5月, 2020 2 次提交
  3. 13 5月, 2020 1 次提交
    • E
      Remove implementation of unchecked_serialize · 6833bf4d
      eileencodes 提交于
      Since we're checking `serializable?` in the new `HomogeneousIn`
      `serialize` will no longer raise an exception. We implemented
      `unchecked_serialize` to avoid raising in these cases, but with some of
      our refactoring we no longer need it.
      
      I discovered this while trying to fix a query in our application that
      was not properly serializing binary columns. I discovered that in at
      least 2 of our active model types we were not calling the correct
      serialization. Since `serialize` wasn't aliased to `unchecked_serialize`
      in `ActiveModel::Type::Binary` and `ActiveModel::Type::Boolean` (I
      didn't check others but pretty sure all the AM Types are broken) the SQL
      was being treated as a `String` and not the correct type.
      
      This caused Rails to incorrectly query by string values. This is
      problematic for columns storing binary data like our emoji columns at
      GitHub. The test added here is an example of how the Binary type was
      broken previously. The SQL should be using the hex values, not the
      string value of "🥦" or other emoji.
      
      We still have the problem `unchecked_serialize` was supposed to fix -
      that `serialize` shouldn't validate data, just convert it. We'll be
      fixing that in a followup PR so for now we should use `serialize` so we
      know all the values are going through the right serialization for their
      SQL.
      6833bf4d
  4. 12 5月, 2020 1 次提交
  5. 11 5月, 2020 1 次提交
    • R
      Add `add_binds` to bulk binds processing for performance · 2b35775f
      Ryuta Kamizono 提交于
      Before with `prepared_statements: true`:
      
      ```
      Warming up --------------------------------------
            where with ids     9.000  i/100ms
      Calculating -------------------------------------
            where with ids     91.992  (± 7.6%) i/s -    459.000  in   5.020817s
      ```
      
      Now with `prepared_statements: true`:
      
      ```
      Warming up --------------------------------------
            where with ids    10.000  i/100ms
      Calculating -------------------------------------
            where with ids     99.393  (± 8.0%) i/s -    500.000  in   5.069280s
      ```
      2b35775f
  6. 10 5月, 2020 1 次提交
    • R
      Should not substitute binds when `prepared_statements: true` · 157f6a6e
      Ryuta Kamizono 提交于
      Before IN clause optimization 70ddb8a7, Active Record had generated an
      SQL with binds when `prepared_statements: true`:
      
      ```ruby
      # prepared_statements: true
      #
      #   SELECT `authors`.* FROM `authors` WHERE `authors`.`id` IN (?, ?, ?)
      #
      # prepared_statements: false
      #
      #   SELECT `authors`.* FROM `authors` WHERE `authors`.`id` IN (1, 2, 3)
      #
      Author.where(id: [1, 2, 3]).to_a
      ```
      
      But now, binds in IN clause is substituted regardless of whether
      `prepared_statements: true` or not:
      
      ```ruby
      # prepared_statements: true
      #
      #   SELECT `authors`.* FROM `authors` WHERE `authors`.`id`IN (1,2,3)
      #
      # prepared_statements: false
      #
      #   SELECT `authors`.* FROM `authors` WHERE `authors`.`id`IN (1,2,3)
      #
      Author.where(id: [1, 2, 3]).to_a
      ```
      
      I suppose that is considered as a regression for the context:
      
      > While I would prefer that we fix/avoid the too-many-parameters
      problem, but I don't like the idea of globally ditching bind params for
      this edge case... we're getting to the point where I'd almost consider
      anything that doesn't use a bind to be a bug.
      
      https://github.com/rails/rails/pull/33844#issuecomment-421000003
      
      This makes binds consider whether `prepared_statements: true` or not
      (i.e. restore the original behavior as before), but still gain that
      optimization when need the substitute binds (`prepared_statements: false`,
      `relation.to_sql`). Even when `prepared_statements: true`, it still
      much faster than before by optimized (bind node less) binds generation.
      
      ```ruby
      class Post < ActiveRecord::Base
      end
      
      ids = (1..1000).each.map do |n|
        Post.create!.id
      end
      
      puts "prepared_statements: #{Post.connection.prepared_statements.inspect}"
      
      Benchmark.ips do |x|
        x.report("where with ids") do
          Post.where(id: ids).to_a
        end
      end
      ```
      
      * Before (200058b0)
      
      `prepared_statements: true`:
      
      ```
      Warming up --------------------------------------
            where with ids     6.000  i/100ms
      Calculating -------------------------------------
            where with ids     63.806  (± 7.8%) i/s -    318.000  in   5.015903s
      ```
      
      `prepared_statements: false`:
      
      ```
      Warming up --------------------------------------
            where with ids     7.000  i/100ms
      Calculating -------------------------------------
            where with ids     73.550  (± 8.2%) i/s -    371.000  in   5.085672s
      ```
      
      * Now with this change
      
      `prepared_statements: true`:
      
      ```
      Warming up --------------------------------------
            where with ids     9.000  i/100ms
      Calculating -------------------------------------
            where with ids     91.992  (± 7.6%) i/s -    459.000  in   5.020817s
      ```
      
      `prepared_statements: false`:
      
      ```
      Warming up --------------------------------------
            where with ids    10.000  i/100ms
      Calculating -------------------------------------
            where with ids    104.335  (± 8.6%) i/s -    520.000  in   5.026425s
      ```
      157f6a6e
  7. 06 5月, 2020 3 次提交
  8. 03 5月, 2020 1 次提交
  9. 02 5月, 2020 3 次提交
    • R
      7669dc21
    • E
      Perf: Improve performance of where when using an array of values · 72fd0bae
      eileencodes 提交于
      A coworker at GitHub found a few months back that if we used
      `santitize_sql` over `where` when we knew the values going into `where`
      it was a lot faster than `where`.
      
      This PR adds a new Arel node type called `HomogenousIn` that will be
      used when Rails knows the values are all homogenous and can therefore
      pick a faster codepath. This new codepath skips some of the required
      processing by `where` to make `wheres` with homogenous arrays faster
      without requiring the application author to know when to use which query
      type.
      
      Using our benchmark code:
      
      ```ruby
      ids = (1..1000).each.map do |n|
        Post.create!.id
      end
      
      Benchmark.ips do |x|
        x.report("where with ids") do
          Post.where(id: ids).to_a
        end
      
        x.report("where with sanitize") do
          Post.where(ActiveRecord::Base.sanitize_sql(["id IN (?)", ids])).to_a
        end
      
        x.compare!
      end
      ```
      
      Before this PR comparing where with a list of IDs to santitize sql:
      
      ```
      Warming up --------------------------------------
            where with ids    11.000  i/100ms
       where with sanitize    17.000  i/100ms
      
      Calculating -------------------------------------
            where with ids    115.733  (± 4.3%) i/s -    583.000  in   5.045828s
       where with sanitize    174.231  (± 4.0%) i/s -    884.000  in   5.081495s
      
      Comparison:
       where with sanitize:      174.2 i/s
            where with ids:      115.7 i/s - 1.51x  slower
      ```
      
      After this PR comparing where with a list of IDs to santitize sql:
      
      ```
      Warming up --------------------------------------
            where with ids    16.000  i/100ms
       where with sanitize    19.000  i/100ms
      
      Calculating -------------------------------------
            where with ids    158.293  (± 6.3%) i/s -    800.000  in   5.072208s
       where with sanitize    169.141  (± 3.5%) i/s -    855.000  in   5.060878s
      
      Comparison:
       where with sanitize:      169.1 i/s
            where with ids:      158.3 i/s - same-ish: difference falls within error
      ```
      Co-authored-by: NAaron Patterson <aaron.patterson@gmail.com>
      72fd0bae
    • E
      Support query attrs and bind params in dot output · ff8f40e7
      eileencodes 提交于
      ff8f40e7
  10. 29 4月, 2020 1 次提交
    • R
      PERF: Improve performance of where when using an array of values · c3a2b54b
      Ryuta Kamizono 提交于
      This is a smaller alternative of performance improvement, without
      refactoring type casting mechanism #39009.
      
      This is relatively a smaller change (but about 40% faster than before),
      so I think this could be easier reviewed without discuss about
      refactoring type casting mechanism.
      
      This just makes `attribute.in(values)` less allocation from an array of
      casted nodes to one casted array node.
      
      ```ruby
      ids = (1..1000).each.map do |n|
        Post.create!.id
      end
      
      Benchmark.ips do |x|
        x.report("where with ids") do
          Post.where(id: ids).to_a
        end
      
        x.report("where with sanitize") do
          Post.where(ActiveRecord::Base.sanitize_sql(["id IN (?)", ids])).to_a
        end
      
        x.compare!
      end
      ```
      
      Before:
      
      ```
      Warming up --------------------------------------
            where with ids     7.000  i/100ms
       where with sanitize    13.000  i/100ms
      
      Calculating -------------------------------------
            where with ids     70.661  (± 5.7%) i/s -    357.000  in   5.072771s
       where with sanitize    130.993  (± 7.6%) i/s -    663.000  in   5.096085s
      
      Comparison:
       where with sanitize:      131.0 i/s
            where with ids:       70.7 i/s - 1.85x  slower
      ```
      
      After:
      
      ```
      Warming up --------------------------------------
            where with ids    10.000  i/100ms
       where with sanitize    13.000  i/100ms
      
      Calculating -------------------------------------
            where with ids     98.174  (± 7.1%) i/s -    490.000  in   5.012851s
       where with sanitize    132.289  (± 8.3%) i/s -    663.000  in   5.052728s
      
      Comparison:
       where with sanitize:      132.3 i/s
            where with ids:       98.2 i/s - 1.35x  slower
      ```
      c3a2b54b
  11. 28 4月, 2020 1 次提交
  12. 27 4月, 2020 1 次提交
    • R
      Deprecate `in_clause_length` in `DatabaseLimits` · c0ca7625
      Ryuta Kamizono 提交于
      `in_clause_length` was added at c5a284f8 to address to Oracle IN clause
      length limitation.
      
      Now `in_clause_length` is entirely integrated in Arel visitor since
      #35838 and #36074.
      
      Since Oracle visitors are the only code that rely on `in_clause_length`.
      so I'd like to remove that from Rails code base, like has removed Oracle
      visitors (#38946).
      c0ca7625
  13. 26 4月, 2020 1 次提交
    • R
      More concise Arel `Or` ast and make `Or` visitor non recursive · 7d3bff1f
      Ryuta Kamizono 提交于
      Before this, 1000 `Or` nodes will raise "stack level too deep" due to
      visiting too deep Arel ast.
      
      This makes more concise Arel `Or` ast and `Or` visitor non recursive if
      `Or` nodes are adjoined, as a result, "stack level too deep" is no
      longer raised.
      
      ```ruby
      class Post < ActiveRecord::Base
      end
      
      posts = (0..500).map { |i| Post.where(id: i) }
      
      Benchmark.ips do |x|
        x.report("inject scopes") { posts.inject(&:or).to_sql }
      end
      ```
      
      Before:
      
      ```
      Warming up --------------------------------------
            where with ids     9.000  i/100ms
      Calculating -------------------------------------
            where with ids     96.126  (± 2.1%) i/s -    486.000  in   5.058960s
      ```
      
      After:
      
      ```
      Warming up --------------------------------------
             inject scopes    10.000  i/100ms
      Calculating -------------------------------------
             inject scopes    101.714  (± 2.9%) i/s -    510.000  in   5.018880s
      ```
      
      Fixes #39032.
      7d3bff1f
  14. 23 4月, 2020 1 次提交
  15. 15 4月, 2020 1 次提交
  16. 14 3月, 2020 1 次提交
    • E
      Refactor fetch_attribute · ce7bbc36
      eileencodes 提交于
      Similar to the work done in #38636, instead of using case statements we
      can make these classes respond to `fetch_attribute`.
      
      New classes can implement `fetch_attribute` instead of adding to the
      case statement, it's more object oriented, and nicer looking.
      Co-authored-by: NAaron Patterson <aaron.patterson@gmail.com>
      ce7bbc36
  17. 10 3月, 2020 1 次提交
  18. 04 3月, 2020 1 次提交
    • E
      Refactor invert predicate · 27fb3563
      eileencodes 提交于
      Instead of doing a case statement here we can have each of the objects
      respond to `invert`. This means that when adding new objects we don't
      need to increase this case statement, it's more object oriented, and
      let's be fair, it looks better too.
      
      Aaron and I stumbled upon this while working on some performance
      work in Arel.
      
      I removed `random_object` from the invert test because we don't support
      random objects. If you pass a random object to Arel, it should raise,
      not be inverted.
      Co-authored-by: NAaron Patterson <aaron.patterson@gmail.com>
      27fb3563
  19. 07 1月, 2020 1 次提交
  20. 01 1月, 2020 1 次提交
    • K
      Allow #nulls_first and #nulls_last in PostgreSQL · 66b19b5d
      Kevin Deisz 提交于
      When using PostgreSQL, it's useful to be able to specify NULLS FIRST and NULLS LAST on ordered columns. With this commit you can do that now, as in:
      
      ```ruby
      User.arel_table[:first_name].desc.nulls_last
      ```
      66b19b5d
  21. 06 11月, 2019 1 次提交
    • Y
      Address `no implicit conversion of Arel::Attributes::Attribute into String` · eb6c8577
      Yasuo Honda 提交于
      https://github.com/rails/rails/commit/4a9ef5e1202cdab1882989eb561b0dc854c9891b triggers this failure,
      then restored the original code for `activerecord/lib/arel/visitors/oracle.rb` .
      
      ```
      $ bundle exec rspec ./spec/active_record/connection_adapters/oracle_enhanced/context_index_spec.rb:92
      ==> Loading config from ENV or use default
      ==> Running specs with MRI version 2.6.5
      ==> Effective ActiveRecord version 6.1.0.alpha
      ... snip ...
      Run options: include {:locations=>{"./spec/active_record/connection_adapters/oracle_enhanced/context_index_spec.rb"=>[92]}}
      F
      
      Failures:
      
        1) OracleEnhancedAdapter context index on single table should create single VARCHAR2 column index
           Failure/Error: expect(Post.contains(:title, word).to_a).to eq([@post2, @post1])
      
           TypeError:
             no implicit conversion of Arel::Attributes::Attribute into String
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:107:in `match?'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:107:in `block (2 levels) in order_hacks'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:106:in `any?'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:106:in `block in order_hacks'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:105:in `any?'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:105:in `order_hacks'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/oracle.rb:8:in `visit_Arel_Nodes_SelectStatement'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/visitor.rb:30:in `visit'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/visitor.rb:11:in `accept'
           # /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb:10:in `accept'
           # /home/yahonda/git/rails/activerecord/lib/arel/visitors/to_sql.rb:18:in `compile'
           # /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:25:in `to_sql_and_binds'
           # /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:63:in `select_all'
           # /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:107:in `select_all'
           # /home/yahonda/git/rails/activerecord/lib/active_record/querying.rb:46:in `find_by_sql'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:826:in `block in exec_queries'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:844:in `skip_query_cache_if_necessary'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:811:in `exec_queries'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:629:in `load'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:250:in `records'
           # /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:245:in `to_ary'
           # ./spec/active_record/connection_adapters/oracle_enhanced/context_index_spec.rb:95:in `block (4 levels) in <top (required)>'
           ... snip ...
      ```
      eb6c8577
  22. 04 10月, 2019 1 次提交
  23. 17 9月, 2019 1 次提交
  24. 06 9月, 2019 1 次提交
  25. 26 8月, 2019 1 次提交
  26. 31 7月, 2019 1 次提交
  27. 29 7月, 2019 1 次提交
    • J
      Enabled matches_regex for MySql · 92c265b3
      James Pearson 提交于
      Previously matches_regex was only availble on PostgreSql, this will enable it for MySql
      
      Usage example:
      
          users = User.arel_table;
          users = User.arel_table; User.where(users[:email].matches_regexp('(.*)\@gmail.com'))
      
      Update activerecord/test/cases/arel/visitors/mysql_test.rb
      Co-Authored-By: NRyuta Kamizono <kamipo@gmail.com>
      92c265b3
  28. 27 7月, 2019 1 次提交
  29. 17 7月, 2019 1 次提交
    • J
      Support beginless ranges in hash conditions. · b1915044
      Josh Goodall 提交于
      Ruby 2.7 introduces beginless ranges (..value and ...value) and as with
      endless ranges we can turn these into inequalities, enabling expressions
      such as
      
          Order.where(created_at: ..1.year.ago)
          User.where(karma: ...0)
      b1915044
  30. 10 7月, 2019 1 次提交
  31. 15 6月, 2019 3 次提交
    • R
      Remove unused `Arel::Attributes.for` · b57b23a3
      Ryuta Kamizono 提交于
      `Arel::Attributes.for` is no longer used since https://github.com/rails/arel/pull/196.
      b57b23a3
    • R
      Remove unused `DepthFirst` visitor · fc38ff6e
      Ryuta Kamizono 提交于
      We only use `ToSQL` visitors in the our codebase, do not use
      `DepthFirst` and `Dot` visitors.
      
      The `DepthFirst` visitor (which was introduced at c86c37e5) is used to
      traverse an Arel (partial) ast with depth first.
      
      Is there any worth to keep that undocumented feature with much code and
      test cases.
      
      This removes that unused `DepthFirst` code and test cases.
      fc38ff6e
    • R
      No allocation `Arel::Visitors::ToSql#visit` · 85b4ba28
      Ryuta Kamizono 提交于
      Each `visit o, collector` allocates one extra array due to
      receiving args by splat array.
      
      https://github.com/rails/rails/blob/2c3332cc4c0fa77dbe2e13e8a792f80fbd8f4ad3/activerecord/lib/arel/visitors/visitor.rb#L27-L29
      
      Currently 1,000 times `User.where(id: 1).to_sql` allocates 13,000
      arrays in `visitor.accept`. This avoids receiving args by splat array,
      it makes `visitor.accept` no array allocation.
      
      ```ruby
      ObjectSpace::AllocationTracer.setup(%i{path line type})
      
      pp ObjectSpace::AllocationTracer.trace {
        1_000.times { User.where(id: 1).to_sql }
      }.select { |k, _| k[2] == :T_ARRAY && k[0].end_with?("visitor.rb", "to_sql.rb") }
      ```
      
      Before (2c3332cc):
      
      ```
      {["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        18,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb",
        11,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/visitor.rb",
        12,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        788,
        :T_ARRAY]=>[3000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        794,
        :T_ARRAY]=>[3000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        156,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        443,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        603,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0],
       ["~/rails/activerecord/lib/arel/visitors/to_sql.rb",
        611,
        :T_ARRAY]=>[1000, 0, 0, 0, 0, 0]}
      ```
      
      After (this change):
      
      ```
      {}
      ```
      85b4ba28
  32. 13 6月, 2019 1 次提交