- 15 12月, 2017 4 次提交
-
-
由 Andrew Kryczka 提交于
Summary: added `ThreadType::BOTTOM_PRIORITY` which is used in the `ThreadStatus` object to indicate the thread is used for bottom-pri compactions. Previously there was a bug where we mislabeled such threads as `ThreadType::LOW_PRIORITY`. Closes https://github.com/facebook/rocksdb/pull/3270 Differential Revision: D6559428 Pulled By: ajkr fbshipit-source-id: 96b1a50a9c19492b1a5fd1b77cf7061a6f9f1d1c
-
由 Orvid King 提交于
Summary: There were a few places where MSVC's implicit truncation warnings were getting triggered, which was causing the MSVC build to fail due to warnings being treated as errors. This resolves the issues by making the truncations in some places explicit, and by making it so there are no truncations of literals. Fixes #3239 Supersedes #3259 Closes https://github.com/facebook/rocksdb/pull/3273 Reviewed By: yiwu-arbug Differential Revision: D6569204 Pulled By: Orvid fbshipit-source-id: c188cf1cf98d9acb6d94b71875041cc81f8ff088
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3264 Differential Revision: D6552768 Pulled By: siying fbshipit-source-id: 6303110aff22f341d5cff41f8d2d4f138a53652d
-
由 Siying Dong 提交于
Summary: NUMBER_BLOCK_DECOMPRESSED and NUMBER_BLOCK_COMPRESSED are not reported unless the stats level contain detailed timers, which is wrong. They are normal counters. Fix it. Closes https://github.com/facebook/rocksdb/pull/3263 Differential Revision: D6552519 Pulled By: siying fbshipit-source-id: 40899ccea7b2856bb39752616657c0bfd432f6f9
-
- 14 12月, 2017 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Add "--use_txn" option to use transactional API in db_stress, default being WRITE_PREPARED policy, which is the main intention of modifying db_stress. It also extend the existing snapshots to verify that before releasing a snapshot a read from it returns the same value as before. Closes https://github.com/facebook/rocksdb/pull/3243 Differential Revision: D6556912 Pulled By: maysamyabandeh fbshipit-source-id: 1ae31465be362d44bd06e635e2e9e49a1da11268
-
- 13 12月, 2017 7 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Currently WriteImplWALOnly simply returns when disableWAL is set. This is an incorrect behavior since it does not allocated the sequence number, which is a side-effect of writing to the WAL. This patch fixes the issue. Closes https://github.com/facebook/rocksdb/pull/3262 Differential Revision: D6550974 Pulled By: maysamyabandeh fbshipit-source-id: 745a83ae8f04e7ca6c8ffb247d6ef16c287c52e7
-
由 Maysam Yabandeh 提交于
Summary: Garbage collect entries from old_commit_map_ when the corresponding snapshots are released. Closes https://github.com/facebook/rocksdb/pull/3247 Differential Revision: D6528478 Pulled By: maysamyabandeh fbshipit-source-id: 15d1566d85d4ac07036bc0dc47418f6c3228d4bf
-
由 Zhongyi Xie 提交于
Summary: This PR addresses the following heavy hitters in `Get` operation by moving calls to `StatisticsImpl::recordTick` from `BlockBasedTable` to `Version::Get` - rocksdb.block.cache.bytes.write - rocksdb.block.cache.add - rocksdb.block.cache.data.miss - rocksdb.block.cache.data.bytes.insert - rocksdb.block.cache.data.add - rocksdb.block.cache.hit - rocksdb.block.cache.data.hit - rocksdb.block.cache.bytes.read The db_bench statistics before and after the change are: |1GB block read|Children |Self |Command |Shared Object |Symbol| |---|---|---|---|---|---| |master: |4.22% |1.31% |db_bench |db_bench |[.] rocksdb::StatisticsImpl::recordTick| |updated: |0.51% |0.21% |db_bench |db_bench |[.] rocksdb::StatisticsImpl::recordTick| | |0.14% |0.14% |db_bench |db_bench |[.] rocksdb::GetContext::record_counters| |1MB block read|Children |Self |Command |Shared Object |Symbol| |---|---|---|---|---|---| |master: |3.48% |1.08% |db_bench |db_bench |[.] rocksdb::StatisticsImpl::recordTick| |updated: |0.80% |0.31% |db_bench |db_bench |[.] rocksdb::StatisticsImpl::recordTick| | |0.35% |0.35% |db_bench |db_bench |[.] rocksdb::GetContext::record_counters| Closes https://github.com/facebook/rocksdb/pull/3172 Differential Revision: D6330532 Pulled By: miasantreble fbshipit-source-id: 2b492959e00a3db29e9437ecdcc5e48ca4ec5741
-
由 Islam AbdelRahman 提交于
Summary: DeleteScheduler::MarkAsTrash() don't handle existing .trash files correctly This cause rocksdb to not being able to delete existing .trash files on restart Closes https://github.com/facebook/rocksdb/pull/3261 Differential Revision: D6548003 Pulled By: IslamAbdelRahman fbshipit-source-id: c3800639412e587a690062c63076a5a08881e0e6
-
由 Yi Wu 提交于
Summary: Some call sites of BlockFetcher create temporary ReadOptions and pass to BlockFetcher. The temporary object will be gone after BlockFetcher construction but BlockFetcher keep its reference, causing stack-use-after-scope. Fixing it. Closes https://github.com/facebook/rocksdb/pull/3258 Differential Revision: D6547152 Pulled By: yiwu-arbug fbshipit-source-id: 6b49e9dd46bb72307f5d8f88ea15faacff35b9bc
-
由 Souvik Banerjee 提交于
Summary: Fixes a compile error on gcc 7.2.1 (-Werror=format-truncation=). Closes https://github.com/facebook/rocksdb/pull/3248 Differential Revision: D6546515 Pulled By: yiwu-arbug fbshipit-source-id: bd78cca63f2af376faceccb1838d2d4cc9208fef
-
由 Yi Wu 提交于
Summary: Add snapshot_checker check whenever we need to check sequence against snapshots and decide what to do with an input key. The changes are related to one of: * compaction filter * single delete * delete at bottom level * merge Closes https://github.com/facebook/rocksdb/pull/3251 Differential Revision: D6537850 Pulled By: yiwu-arbug fbshipit-source-id: 3faba40ed5e37779f4a0cb7ae78af9546659c7f2
-
- 12 12月, 2017 8 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3256 Differential Revision: D6541714 Pulled By: siying fbshipit-source-id: 40efd89b68587a9d58cfe6f4eebd771c2d9f1542
-
由 Zhongyi Xie 提交于
Summary: since [SetOptions](https://github.com/facebook/rocksdb/blob/master/db/db_impl.cc#L494) is not supported in ROCKSDB_LITE Right now unit test under lite is broken Closes https://github.com/facebook/rocksdb/pull/3253 Differential Revision: D6539428 Pulled By: miasantreble fbshipit-source-id: 13172b8ecbd75682330726498ea198969bc3e637
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3252 Differential Revision: D6539373 Pulled By: siying fbshipit-source-id: ce7c3d3fe625852179055295da9cf7bc80755025
-
由 Siying Dong 提交于
Summary: Divide ReadBlockContents() to multiple sub-functions. Maintaining the input and intermediate data in a new class BlockFetcher. I hope in general it makes the code easier to maintain. Another motivation to do it is to clearly divide the logic before file reading and after file reading. The refactor will help us evaluate how can we make I/O async in the future. Closes https://github.com/facebook/rocksdb/pull/3244 Differential Revision: D6520983 Pulled By: siying fbshipit-source-id: 338d90bc0338472d46be7a7682028dc9114b12e9
-
由 Yi Wu 提交于
Summary: A data race is caught by tsan_crash test between compaction and DropColumnFamily: https://gist.github.com/yiwu-arbug/5a2b4baae05eeb99ae1719b650f30a44 Compaction checks if the column family has been dropped on each key input, while user can issue DropColumnFamily which updates cfd->dropped_, causing the data race. Fixing it by making cfd->dropped_ an atomic. Closes https://github.com/facebook/rocksdb/pull/3250 Differential Revision: D6535991 Pulled By: yiwu-arbug fbshipit-source-id: 5571df020beae7fa7db6fff5ad0d598f49962895
-
由 Zhongyi Xie 提交于
Summary: Let me know if more test coverage is needed Closes https://github.com/facebook/rocksdb/pull/3213 Differential Revision: D6457165 Pulled By: miasantreble fbshipit-source-id: 3f944abff28aa7775237f1c4f61c64ccbad4eea9
-
由 Yi Wu 提交于
Summary: Refactor BlobDB open logic. List of changes: Major: * On reopen, mark blob files found as immutable, do not use them for writing new keys. * Not to scan the whole file to find file footer. Instead just seek to the end of the file and try to read footer. Minor: * Move most of the real logic from blob_db.cc to blob_db_impl.cc. * Not to hold shared_ptr of event listeners in global maps in blob_db.cc * Some changes to BlobFile interface. * Improve logging and error handling. Closes https://github.com/facebook/rocksdb/pull/3246 Differential Revision: D6526147 Pulled By: yiwu-arbug fbshipit-source-id: 9dc4cdd63359a2f9b696af817086949da8d06952
-
由 Prashant D 提交于
Summary: db/compaction_job.cc: ReportStartedCompaction(compaction); CID 1419863 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 2. uninit_member: Non-static class member bottommost_level_ is not initialized in this constructor nor in any functions that it calls. db/compaction_picker_universal.cc: 7struct InputFileInfo { 2. uninit_member: Non-static class member level is not initialized in this constructor nor in any functions that it calls. CID 1405355 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 4. uninit_member: Non-static class member index is not initialized in this constructor nor in any functions that it calls. 38 InputFileInfo() : f(nullptr) {} db/dbformat.h: ParsedInternalKey() 84 : sequence(kMaxSequenceNumber) // Make code analyzer happy CID 1168095 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 2. uninit_member: Non-static class member type is not initialized in this constructor nor in any functions that it calls. 85 {} // Intentionally left uninitialized (for speed) Closes https://github.com/facebook/rocksdb/pull/3091 Differential Revision: D6534558 Pulled By: yiwu-arbug fbshipit-source-id: 5ada975956196d267b3f149386842af71eda7553
-
- 08 12月, 2017 5 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3242 Differential Revision: D6514255 Pulled By: ajkr fbshipit-source-id: c39fa8e745866b052649d02bf339e794d77e96a3
-
由 Dmitri Smirnov 提交于
Summary: Fix a race condition when we create a thread and immediately destroy This case should be supported. What happens is that the thread function needs the Data instance to actually run but has no shared ownership and must rely on the WindowsThread instance to continue existing. To address this we change unique_ptr to shared_ptr and then acquire an additional refcount for the threadproc which destroys it just before the thread exit. We choose to allocate shared_ptr instance on the heap as this allows the original thread to continue w/o waiting for the new thread to start running. Closes https://github.com/facebook/rocksdb/pull/3240 Differential Revision: D6511324 Pulled By: yiwu-arbug fbshipit-source-id: 4633ff7996daf4d287a9fe34f60c1dd28cf4ff36
-
由 Prashant D 提交于
Summary: db/version_builder.cc: 117 base_vstorage_->InternalComparator(); CID 1351713 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 2. uninit_member: Non-static class member field level_zero_cmp_.internal_comparator is not initialized in this constructor nor in any functions that it calls. db/version_edit.h: 145 FdWithKeyRange() 146 : fd(), 147 smallest_key(), 148 largest_key() { CID 1418254 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 2. uninit_member: Non-static class member file_metadata is not initialized in this constructor nor in any functions that it calls. 149 } db/version_set.cc: 120 } CID 1322789 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 4. uninit_member: Non-static class member curr_file_level_ is not initialized in this constructor nor in any functions that it calls. 121 } db/write_batch.cc: 939 assert(cf_mems_); CID 1419862 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 3. uninit_member: Non-static class member rebuilding_trx_seq_ is not initialized in this constructor nor in any functions that it calls. 940 } Closes https://github.com/facebook/rocksdb/pull/3092 Differential Revision: D6505666 Pulled By: yiwu-arbug fbshipit-source-id: fd2c68948a0280772691a419d72ac7e190951d86
-
由 Prashant D 提交于
Summary: table/block.cc: 420 } CID 1396127 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 7. uninit_member: Non-static class member restart_offset_ is not initialized in this constructor nor in any functions that it calls. 421} table/block_based_table_builder.cc: CID 1418259 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 7. uninit_member: Non-static class member compressed_cache_key_prefix_size is not initialized in this constructor nor in any functions that it calls. table/block_based_table_reader.h: 3. uninit_member: Non-static class member index_type is not initialized in this constructor nor in any functions that it calls. CID 1396147 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 5. uninit_member: Non-static class member hash_index_allow_collision is not initialized in this constructor nor in any functions that it calls. 413 global_seqno(kDisableGlobalSequenceNumber) {} 414 table/cuckoo_table_reader.cc: 55 if (hash_funs == user_props.end()) { 56 status_ = Status::Corruption("Number of hash functions not found"); 5. uninit_member: Non-static class member is_last_level_ is not initialized in this constructor nor in any functions that it calls. 7. uninit_member: Non-static class member identity_as_first_hash_ is not initialized in this constructor nor in any functions that it calls. 9. uninit_member: Non-static class member use_module_hash_ is not initialized in this constructor nor in any functions that it calls. 11. uninit_member: Non-static class member num_hash_func_ is not initialized in this constructor nor in any functions that it calls. 13. uninit_member: Non-static class member key_length_ is not initialized in this constructor nor in any functions that it calls. 15. uninit_member: Non-static class member user_key_length_ is not initialized in this constructor nor in any functions that it calls. 17. uninit_member: Non-static class member value_length_ is not initialized in this constructor nor in any functions that it calls. 19. uninit_member: Non-static class member bucket_length_ is not initialized in this constructor nor in any functions that it calls. 21. uninit_member: Non-static class member cuckoo_block_size_ is not initialized in this constructor nor in any functions that it calls. 23. uninit_member: Non-static class member cuckoo_block_bytes_minus_one_ is not initialized in this constructor nor in any functions that it calls. CID 1322785 (#2 of 2): Uninitialized scalar field (UNINIT_CTOR) 25. uninit_member: Non-static class member table_size_ is not initialized in this constructor nor in any functions that it calls. 57 return; table/plain_table_index.h: 2. uninit_member: Non-static class member index_size_ is not initialized in this constructor nor in any functions that it calls. CID 1322801 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 4. uninit_member: Non-static class member sub_index_size_ is not initialized in this constructor nor in any functions that it calls. 128 huge_page_tlb_size_(huge_page_tlb_size) {} 129 Closes https://github.com/facebook/rocksdb/pull/3113 Differential Revision: D6505719 Pulled By: yiwu-arbug fbshipit-source-id: 38f44d8f9dfefb4c2e25d83b8df25a5201c75618
-
由 Andrew Kryczka 提交于
Summary: error message was ``` ==3095==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffd18216c40 at pc 0x0000005edda1 bp 0x7ffd18215550 sp 0x7ffd18214d00 ... Address 0x7ffd18216c40 is located in stack of thread T0 at offset 1952 in frame #0 internal_repo_rocksdb/db_compaction_test.cc:1520 rocksdb::DBCompactionTest_DeleteFileRangeFileEndpointsOverlapBug_Test::TestBody() ``` It was unsafe to have slices referring to the temporary string objects' buffers, as those strings were destroyed before the slices were used. Fixed it by assigning the strings returned by `Key()` to local variables. Closes https://github.com/facebook/rocksdb/pull/3238 Differential Revision: D6507864 Pulled By: ajkr fbshipit-source-id: dd07de1a0070c6748c1ab4f3d7bd31f9a81889d0
-
- 07 12月, 2017 10 次提交
-
-
由 Yi Wu 提交于
Summary: clang-analyzer complaint about db_ being nullptr, but it couldn't be because it checks exec_stats before proceed. Add an assert to get around the false-positive. Test Plan `make analyze` Closes https://github.com/facebook/rocksdb/pull/3236 Differential Revision: D6505417 Pulled By: yiwu-arbug fbshipit-source-id: e5b65764ea994dd9e4bab3e697b97dc70dc22cab
-
由 Sagar Vemuri 提交于
Summary: `GetCurrentTime()` is used to populate `creation_time` table property during flushes and compactions. It is safe to ignore `GetCurrentTime()` failures here but they should be logged. (Note that `creation_time` property was introduced as part of TTL-based FIFO compaction in #2480.) Tes Plan: `make check` Closes https://github.com/facebook/rocksdb/pull/3231 Differential Revision: D6501935 Pulled By: sagar0 fbshipit-source-id: 376adcf4ab801d3a43ec4453894b9a10909c8eb6
-
由 Sagar Vemuri 提交于
Summary: Adding more BlobDB db_bench options which are needed for benchmarking. Closes https://github.com/facebook/rocksdb/pull/3230 Differential Revision: D6500711 Pulled By: sagar0 fbshipit-source-id: 91d63122905854ef7c9148a0235568719146e6c5
-
由 Andrew Kryczka 提交于
Summary: I browsed through the history since 5.9 was released and found some changes worth mentioning in HISTORY.md. Closes https://github.com/facebook/rocksdb/pull/3237 Differential Revision: D6506472 Pulled By: ajkr fbshipit-source-id: 627ce9f94ca33df9f0f231a9c5ced3624b05506c
-
由 Andrew Kryczka 提交于
Summary: Fix for #2833. - In `DeleteFilesInRange`, use `GetCleanInputsWithinInterval` instead of `GetOverlappingInputs` to make sure we get a clean cut set of files to delete. - In `GetCleanInputsWithinInterval`, support nullptr as `begin_key` or `end_key`. - In `GetOverlappingInputsRangeBinarySearch`, move the assertion for non-empty range away from `ExtendFileRangeWithinInterval`, which should be allowed to return an empty range (via `end_index < begin_index`). Closes https://github.com/facebook/rocksdb/pull/2843 Differential Revision: D5772387 Pulled By: ajkr fbshipit-source-id: e554e8461823c6be82b21a9262a2da02b3957881
-
由 Yi Wu 提交于
Summary: Rename shadow variable name db_impl. Fixing #3227 Closes https://github.com/facebook/rocksdb/pull/3235 Differential Revision: D6504051 Pulled By: yiwu-arbug fbshipit-source-id: 186c9378dabb11f8d6db56f45c95cc3b029fcb88
-
由 Yi Wu 提交于
Summary: This is to fix tools/check_format_compatible.sh. The tool try to open old versions of rocksdb with the provided options file. When options file is missing (e.g. rocksdb 2.2), it should still proceed with default options. Closes https://github.com/facebook/rocksdb/pull/3232 Differential Revision: D6503955 Pulled By: yiwu-arbug fbshipit-source-id: e44cfcce7ddc7d12cf83466ed3f3fe7624aa78b8
-
由 Yi Wu 提交于
Summary: Add recent versions for format compatible test. We should probably update the script to auto include available versions (by looking at include/rocksdb/versions.h and deduce branch names), but we can do it later. Closes https://github.com/facebook/rocksdb/pull/3233 Differential Revision: D6503631 Pulled By: yiwu-arbug fbshipit-source-id: e2b01d1ef6e784ff6ffa1bd75d741755e3c69a8c
-
由 Yi Wu 提交于
Summary: Since #1665, on merge error, iterator will be set to corrupted status, but it doesn't invalidate the iterator. Fixing it. Closes https://github.com/facebook/rocksdb/pull/3226 Differential Revision: D6499094 Pulled By: yiwu-arbug fbshipit-source-id: 80222930f949e31f90a6feaa37ddc3529b510d2c
-
由 Maysam Yabandeh 提交于
Summary: Augment the existing MySQLStyleTransactionTest to check for more core case scenarios. The changes showed effective in revealing the bugs reported in https://github.com/facebook/rocksdb/pull/3205 and https://github.com/facebook/rocksdb/pull/3101 Closes https://github.com/facebook/rocksdb/pull/3222 Differential Revision: D6476862 Pulled By: maysamyabandeh fbshipit-source-id: 5068497702d67ffc206a58ed96f8578fbb510137
-
- 06 12月, 2017 2 次提交
-
-
由 Flarnie Marchan 提交于
Summary: In the past Facebook didn't promote including a Code of Conduct when creating new projects, and many projects skipped this important document. Let's fix it. :) **why make this change?:** Facebook Open Source provides a Code of Conduct statement for all projects to follow, to promote a welcoming and safe open source community. Exposing the COC via a separate markdown file is a standard being promoted by Github via the Community Profile in order to meet their Open Source Guide's recommended community standards. As you can see, adding this file will improve [the rocksdb community profile](https://github.com/facebook/rocksdb/community) checklist and increase the visibility of our COC. **test plan:** Viewing it on my branch - <img width="1008" alt="screen shot 2017-12-03 at 5 05 45 pm" src="https://user-images.githubusercontent.com/1114467/33532198-66012a56-d84c-11e7-8fab-29ed410bd600.png"> <img width="1015" alt="screen shot 2017-12-03 at 5 05 59 pm" src="https://user-images.githubusercontent.com/1114467/33532199-661813d8-d84c-11e7-941e-94754dd481e5.png"> **issue:** internal task t23481323 Closes https://github.com/facebook/rocksdb/pull/3219 Reviewed By: yiwu-arbug Differential Revision: D6494234 Pulled By: flarnie fbshipit-source-id: 55b59db335cc5546f3a1c968322b9281a3dc3aaf
-
由 Adam Novak 提交于
Summary: Should fix #3036. Closes https://github.com/facebook/rocksdb/pull/3042 Differential Revision: D6452921 Pulled By: sagar0 fbshipit-source-id: eaf11e43fee1f8747006530cfc0c7a358f1c2f0f
-
- 04 12月, 2017 1 次提交
-
-
由 Alex Robinson 提交于
Summary: The Options header file recommends using max_background_jobs rather than directly setting max_background_compactions or max_background_flushes. I've personally seen a performance problem where stalls were happening because the one background flushing thread was blocked that was fixed by this change - https://github.com/cockroachdb/cockroach/issues/19699#issuecomment-347672485 Closes https://github.com/facebook/rocksdb/pull/3208 Differential Revision: D6473178 Pulled By: ajkr fbshipit-source-id: 67c892ceb7b1909d251492640cb15a0f2262b7ed
-
- 02 12月, 2017 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: We should use it if available otherwise the tools builds never work. Thanks to #3212, we can set -DGFLAGS=1 and it'll be independent of the namespace with which gflags was compiled. Closes https://github.com/facebook/rocksdb/pull/3214 Differential Revision: D6462214 Pulled By: ajkr fbshipit-source-id: db4e5f1b905322e3119554a9d01b57532c499384
-
由 Andrew Kryczka 提交于
Summary: I started adding gflags support for cmake on linux and got frustrated that I'd need to duplicate the build_detect_platform logic, which determines namespace based on attempting compilation. We can do it differently -- use the GFLAGS_NAMESPACE macro if available, and if not, that indicates it's an old gflags version without configurable namespace so we can simply hardcode "google". Closes https://github.com/facebook/rocksdb/pull/3212 Differential Revision: D6456973 Pulled By: ajkr fbshipit-source-id: 3e6d5bde3ca00d4496a120a7caf4687399f5d656
-