1. 14 7月, 2021 1 次提交
    • wu-sheng's avatar
      Remove synchronous persistence mechanism in API level and ElasticSearch implementation (#7293) · de975c7e
      wu-sheng 提交于
      * Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
      * Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
      * Remove `syncBulkActions` in ElasticSearch storage option.
      * Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
      * Increase the flush interval of ElasticSearch indices to 15s(from 10s)
      
      Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
      1. https://github.com/elastic/elasticsearch/issues/50199
      2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124
      
      Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.
      
      Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
      de975c7e
  2. 13 7月, 2021 1 次提交
  3. 09 7月, 2021 1 次提交
  4. 08 7月, 2021 1 次提交
  5. 07 7月, 2021 1 次提交
  6. 06 7月, 2021 1 次提交
  7. 05 7月, 2021 1 次提交
  8. 03 7月, 2021 1 次提交
  9. 01 7月, 2021 2 次提交
  10. 30 6月, 2021 1 次提交
    • wu-sheng's avatar
      Optimize L1 aggregation, enhance gRPC client between L1 and L2 aggregation. (#7206) · 50a0bcbf
      wu-sheng 提交于
      1.0Performance: Add L1 aggregation flush period, which reduces the CPU load and helps young GC.
      
      2. Replace do not direct send after the first aggregation to reduce the network #6400.
      
      3. Enhance the DataCarrier to notify the consumer in no enqueue event in short term.
      
      4. L1 aggregation flush period still works even no further metrics generated, powered by <3>
      
      5. Fix gRPC remote client OOM. The concurrency control mechanism failed.
      50a0bcbf
  11. 29 6月, 2021 1 次提交
  12. 28 6月, 2021 1 次提交
  13. 22 6月, 2021 2 次提交
  14. 20 6月, 2021 1 次提交
  15. 19 6月, 2021 1 次提交
  16. 17 6月, 2021 1 次提交
  17. 11 6月, 2021 2 次提交
  18. 28 5月, 2021 2 次提交
  19. 27 5月, 2021 2 次提交
  20. 20 5月, 2021 1 次提交
  21. 16 5月, 2021 1 次提交
  22. 13 5月, 2021 1 次提交
  23. 07 5月, 2021 2 次提交
  24. 27 4月, 2021 1 次提交
  25. 26 4月, 2021 1 次提交
  26. 25 4月, 2021 1 次提交
  27. 23 4月, 2021 1 次提交
  28. 19 4月, 2021 2 次提交
  29. 17 4月, 2021 1 次提交
  30. 15 4月, 2021 1 次提交
  31. 14 4月, 2021 1 次提交
  32. 13 4月, 2021 1 次提交
  33. 08 4月, 2021 1 次提交