Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 06 4月, 2018 1 次提交
    • Z
      fix build for rocksdb lite · c827b2dc
      由 Zhongyi Xie 提交于 
      Summary:
currently rocksdb lite build fails due to the following errors:
> db/db_sst_test.cc:29:51: error: ‘FlushJobInfo’ does not name a type
   virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
                                                   ^
db/db_sst_test.cc:29:16: error: ‘virtual void rocksdb::FlushedFileCollector::OnFlushCompleted(rocksdb::DB*, const int&)’ marked ‘override’, but does not override
   virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
                ^
db/db_sst_test.cc:24:7: error: ‘class rocksdb::FlushedFileCollector’ has virtual functions and accessible non-virtual destructor [-Werror=non-virtual-dtor]
 class FlushedFileCollector : public EventListener {
       ^
db/db_sst_test.cc: In member function ‘virtual void rocksdb::FlushedFileCollector::OnFlushCompleted(rocksdb::DB*, const int&)’:
db/db_sst_test.cc:31:35: error: request for member ‘file_path’ in ‘info’, which is of non-class type ‘const int’
     flushed_files_.push_back(info.file_path);
                                   ^
cc1plus: all warnings being treated as errors
make: *** [db/db_sst_test.o] Error 1
Closes https://github.com/facebook/rocksdb/pull/3676

Differential Revision: D7493006

Pulled By: miasantreble

fbshipit-source-id: 77dff0a5b23e27db51be9b9798e3744e6fdec64f
      c827b2dc
  3. 03 4月, 2018 1 次提交
  5. 14 3月, 2018 1 次提交
    • A
      fix flaky DBSSTTest.DeleteSchedulerMultipleDBPaths · 2256dab1
      由 Andrew Kryczka 提交于 
      Summary:
I landed #3544 which made this test flaky. The reason was the files scheduled for deletion sometimes went through the trash-marking process, and sometimes were deleted directly. Our counter only bumped on the former code path, so if the latter code path was used, we'd miss counting a file deleted by deletion scheduler. This PR also bumps the counter in the latter code path.
Closes https://github.com/facebook/rocksdb/pull/3593

Differential Revision: D7226173

Pulled By: yiwu-arbug

fbshipit-source-id: 81ab44c60834df6ff88db1d73ea34e26c6e93c39
      2256dab1
  7. 09 3月, 2018 1 次提交
  9. 07 3月, 2018 2 次提交
    • A
      Disallow compactions if there isn't enough free space · 0a3db28d
      由 amytai 提交于 
      Summary:
This diff handles cases where compaction causes an ENOSPC error.
This does not handle corner cases where another background job is started while compaction is running, and the other background job triggers ENOSPC, although we do allow the user to provision for these background jobs with SstFileManager::SetCompactionBufferSize.
It also does not handle the case where compaction has finished and some other background job independently triggers ENOSPC.

Usage: Functionality is inside SstFileManager. In particular, users should set SstFileManager::SetMaxAllowedSpaceUsage, which is the reference highwatermark for determining whether to cancel compactions.
Closes https://github.com/facebook/rocksdb/pull/3449

Differential Revision: D7016941

Pulled By: amytai

fbshipit-source-id: 8965ab8dd8b00972e771637a41b4e6c645450445
      0a3db28d
    • A
      support multiple db_paths in SstFileManager · 6a3eebba
      由 Andrew Kryczka 提交于 
      Summary:
Now that files scheduled for deletion are kept in the same directory, we don't need to constrain deletion scheduler to `db_paths[0]`. Previously this was done because there was a separate trash directory, and this constraint prevented files from being accidentally copied to another filesystem when they're scheduled for deletion.
Closes https://github.com/facebook/rocksdb/pull/3544

Differential Revision: D7093786

Pulled By: ajkr

fbshipit-source-id: 202f5c92d925eafebec1281fb95bb5828d33414f
      6a3eebba
  11. 13 12月, 2017 1 次提交
  13. 18 11月, 2017 1 次提交
  15. 28 10月, 2017 2 次提交
  17. 24 10月, 2017 1 次提交
    • Y
      Add DB::Properties::kEstimateOldestKeyTime · 66a2c44e
      由 Yi Wu 提交于 
      Summary:
With FIFO compaction we would like to get the oldest data time for monitoring. The problem is we don't have timestamp for each key in the DB. As an approximation, we expose the earliest of sst file "creation_time" property.

My plan is to override the property with a more accurate value with blob db, where we actually have timestamp.
Closes https://github.com/facebook/rocksdb/pull/2842

Differential Revision: D5770600

Pulled By: yiwu-arbug

fbshipit-source-id: 03833c8f10bbfbee62f8ea5c0d03c0cafb5d853a
      66a2c44e
  19. 22 7月, 2017 2 次提交
  21. 16 7月, 2017 1 次提交
  23. 13 6月, 2017 1 次提交
  25. 28 4月, 2017 1 次提交
  27. 22 4月, 2017 1 次提交
  29. 05 4月, 2017 1 次提交
    • A
      Level-based L0->L0 compaction · d659faad
      由 Andrew Kryczka 提交于 
      Summary:
Level-based L0->L0 compaction operates on spans of files that aren't currently being compacted. It reduces the number of L0 files, thus making write stall conditions harder to reach.

- L0->L0 is triggered when base level is unavailable due to pending compactions
- L0->L0 always outputs one file of at most `max_level0_burst_file_size` bytes.
- Subcompactions are disabled for L0->L0 since we want to output one file.
- Input files are chosen as the longest span of available files that will fit within the size limit. This minimizes number of files in L0.
Closes https://github.com/facebook/rocksdb/pull/2027

Differential Revision: D4760318

Pulled By: ajkr

fbshipit-source-id: 9d07183
      d659faad
  31. 17 3月, 2017 1 次提交
  33. 08 3月, 2017 1 次提交
    • L
      fix db_sst_test flakiness · 72202962
      由 Leonidas Galanis 提交于 
      Summary:
db_sst_test had been flaky occasionally in the following way: reached_max_space_on_compaction can in very rare cases be 0. This happens when the limit on maximum allowable space set using SetMaxAllowedSpaceUsage is hit during flush for all test db sizes (1,2,4,8 and 10MB).The fix clears the error returned when the the space limit is reached during flush. This ensures that the compaction call back will always be called. The runtime is increased slightly because the 1MB loop writes more data and hits the limit during multiple flushes until compaction is scheduled.
Closes https://github.com/facebook/rocksdb/pull/1861

Differential Revision: D4557396

Pulled By: lgalanis

fbshipit-source-id: ff778d1
      72202962
  35. 24 1月, 2017 1 次提交
  37. 22 12月, 2016 1 次提交
  39. 22 11月, 2016 1 次提交
  41. 25 10月, 2016 1 次提交
    • S
      DBSSTTest.RateLimitedDelete: not to use real clock · 24495186
      由 sdong 提交于 
      Summary: Using real clock causes failures of DBSSTTest.RateLimitedDelete in some cases. Turn away from the real time. Use fake time instead.

Test Plan: Run the tests and all existing tests.

Reviewers: yiwu, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D65145
      24495186
  43. 08 9月, 2016 1 次提交
  45. 03 9月, 2016 1 次提交
  47. 20 8月, 2016 1 次提交
  49. 16 8月, 2016 1 次提交
  51. 10 8月, 2016 1 次提交
  53. 26 7月, 2016 1 次提交
    • Y
      Fix flaky DBSSTTEST::DeleteObsoleteFilesPendingOutputs · ae0ad719
      由 Yi Wu 提交于 
      Summary: The test is flaky on Travis in osx environment. The background flush the test wanting to block can run behind the L2 manual compaction, making the test actually blocking the L2 compaction and won't able to proceed.

Test Plan: Test run on travis

Reviewers: kradhakrishnan, sdong, andrewkr, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D61101
      ae0ad719
  55. 21 7月, 2016 1 次提交
  57. 12 7月, 2016 2 次提交
    • A
      fix test failure · 816ae098
      由 Aaron Gao 提交于 
      Summary: fix Rocksdb Unit Test USER_FAILURE

Test Plan: make all check -j64

Reviewers: sdong, andrewkr

Reviewed By: andrewkr

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D60603
      816ae098
    • A
      update DB::AddFile to ingest list of sst files · 8e6b38d8
      由 Aaron Gao 提交于 
      Summary:
DB::AddFile(std::string file_path) API that allow them to ingest an SST file created using SstFileWriter
We want to update this interface to be able to accept a list of files that will be ingested, DB::AddFile(std::vector<std::string> file_path_list).

Test Plan:
Add test case `AddExternalSstFileList` in `DBSSTTest`. To make sure:
1. files key ranges are not overlapping with each other
2. each file key range dont overlap with the DB key range
3. make sure no snapshots are held

Reviewers: andrewkr, sdong, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D58587
      8e6b38d8
  59. 22 6月, 2016 1 次提交
  61. 18 5月, 2016 1 次提交
    • A
      [rocksdb] make more options dynamic · 43afd72b
      由 Aaron Gao 提交于 
      Summary:
make more ColumnFamilyOptions dynamic:
- compression
- soft_pending_compaction_bytes_limit
- hard_pending_compaction_bytes_limit
- min_partial_merge_operands
- report_bg_io_stats
- paranoid_file_checks

Test Plan:
Add sanity check in `db_test.cc` for all above options except for soft_pending_compaction_bytes_limit and hard_pending_compaction_bytes_limit.
All passed.

Reviewers: andrewkr, sdong, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D57519
      43afd72b
  63. 20 4月, 2016 1 次提交
    • I
      Fix DBTest.RateLimitedDelete flakiness · b95510dd
      由 Islam AbdelRahman 提交于 
      Summary: We need to enable sync_point processing before creating the SstFileManager to ensure that we are holding the bg delete scheduler thread from running

Test Plan:
run the test
debug using printf

Reviewers: sdong, yhchiang, yiwu, andrewkr

Reviewed By: andrewkr

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D56871
      b95510dd
  65. 19 4月, 2016 1 次提交
    • Y
      Split db_test.cc · 792762c4
      由 Yi Wu 提交于 
      Summary: Split db_test.cc into several files. Moving several helper functions into DBTestBase.

Test Plan: make check

Reviewers: sdong, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: dhruba, andrewkr, kradhakrishnan, yhchiang, leveldb, sdong

Differential Revision: https://reviews.facebook.net/D56715
      792762c4