- 11 12月, 2019 4 次提交
-
-
由 Yanqin Jin 提交于
Summary: For more portability, switch from readdir to Env::GetChildren() in ldb's manifest_dump subcommand. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6139 Test Plan: ``` $make check ``` Manually check ldb command. Differential Revision: D18898197 Pulled By: riversand963 fbshipit-source-id: 92afca379e9fbe78ab70b2eb40d127daad8df5e2
-
由 sdong 提交于
Summary: Right now, in db_stress, compact range is simply executed without any immediate data validation. Add a simply validation which compares hash for all keys within the compact range to stay the same against the same snapshot before and after the compaction. Also, randomly tune most knobs of CompactRangeOptions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6140 Test Plan: Run db_stress with "--compact_range_one_in=2000 --compact_range_width=100000000" for a while. Manually ingest some hacky code and observe the error path. Differential Revision: D18900230 fbshipit-source-id: d96e75bc8c38dd5ec702571ffe7cf5f4ea93ee10
-
由 Jermy Li 提交于
Summary: Error message when running `make` on Mac OS with master branch (v6.6.0): ``` $ make $DEBUG_LEVEL is 1 Makefile:168: Warning: Compiling in debug mode. Don't use the resulting binary in production third-party/folly/folly/synchronization/WaitOptions.cpp:6:10: fatal error: 'folly/synchronization/WaitOptions.h' file not found #include <folly/synchronization/WaitOptions.h> ^ 1 error generated. third-party/folly/folly/synchronization/ParkingLot.cpp:6:10: fatal error: 'folly/synchronization/ParkingLot.h' file not found #include <folly/synchronization/ParkingLot.h> ^ 1 error generated. third-party/folly/folly/synchronization/DistributedMutex.cpp:6:10: fatal error: 'folly/synchronization/DistributedMutex.h' file not found #include <folly/synchronization/DistributedMutex.h> ^ 1 error generated. third-party/folly/folly/synchronization/AtomicNotification.cpp:6:10: fatal error: 'folly/synchronization/AtomicNotification.h' file not found #include <folly/synchronization/AtomicNotification.h> ^ 1 error generated. third-party/folly/folly/detail/Futex.cpp:6:10: fatal error: 'folly/detail/Futex.h' file not found #include <folly/detail/Futex.h> ^ 1 error generated. GEN util/build_version.cc $DEBUG_LEVEL is 1 Makefile:168: Warning: Compiling in debug mode. Don't use the resulting binary in production third-party/folly/folly/synchronization/WaitOptions.cpp:6:10: fatal error: 'folly/synchronization/WaitOptions.h' file not found #include <folly/synchronization/WaitOptions.h> ^ 1 error generated. third-party/folly/folly/synchronization/ParkingLot.cpp:6:10: fatal error: 'folly/synchronization/ParkingLot.h' file not found #include <folly/synchronization/ParkingLot.h> ^ 1 error generated. third-party/folly/folly/synchronization/DistributedMutex.cpp:6:10: fatal error: 'folly/synchronization/DistributedMutex.h' file not found #include <folly/synchronization/DistributedMutex.h> ^ 1 error generated. third-party/folly/folly/synchronization/AtomicNotification.cpp:6:10: fatal error: 'folly/synchronization/AtomicNotification.h' file not found #include <folly/synchronization/AtomicNotification.h> ^ 1 error generated. third-party/folly/folly/detail/Futex.cpp:6:10: fatal error: 'folly/detail/Futex.h' file not found #include <folly/detail/Futex.h> ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6145 Differential Revision: D18910812 fbshipit-source-id: 5a4475466c2d0601657831a0b48d34316b2f0816
-
由 Peter Dillinger 提交于
Summary: Especially with non-integral bits/key now supported, db_crashtest should vary the bloom_bits configuration. The probabilities look like this: 1/2 chance of a uniform int from 0 to 19. This includes overall 1/40 chance of 0 which disables the bloom filter. 1/2 chance of a float from a lognormal distribution with a median of 10. This always produces positive values but with a decent chance of < 1 (overall ~1/40) or > 100 (overall ~1/40), the enforced/coerced implementation limits. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6103 Test Plan: start 'make blackbox_crash_test' several times and look at configuration output Differential Revision: D18734877 Pulled By: pdillinger fbshipit-source-id: 4a38cb057d3b3fc1327f93199f65b9a9ffbd7316
-
- 10 12月, 2019 7 次提交
-
-
由 sdong 提交于
Summary: Formatter somehow complains some recent lines changed. Apply them to make the formatter happy. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6138 Test Plan: See CI passes. Differential Revision: D18895950 fbshipit-source-id: 7d1696cf3e3a682bc10a30cdca748a23c6565255
-
由 sdong 提交于
Summary: Two changes: 1. Prevent static variables in a header file 2. Add "override" keyword when virtual functions are overridden. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6137 Test Plan: Build db_stress with or without LITE. Differential Revision: D18892007 fbshipit-source-id: 295356427a34473b23ed36d6ed4ef3ae35a32db0
-
由 Peter Dillinger 提交于
Summary: Add overrides needed in FilterPolicy wrapper to fix rocksdb_filterpolicy_create_bloom_full (see issue https://github.com/facebook/rocksdb/issues/6129). Re-enabled assertion in BloomFilterPolicy::CreateFilter that was being violated. Expanded c_test to identify Bloom filter implementations by FP counts. (Without the fix, updated test will trigger assertion and fail otherwise without the assertion.) Fixes https://github.com/facebook/rocksdb/issues/6129 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6132 Test Plan: updated c_test, also run under valgrind. Differential Revision: D18864911 Pulled By: pdillinger fbshipit-source-id: 08e81d7b5368b08e501cd402ef5583f2650c19fa
-
由 sdong 提交于
Summary: thread_local_test now fails because it asserts no thread local instance is created when the test started. However, right now a thread local instance might be created when creating PosixEnv as a static variable. Fix the test by relaxing the assumption of starting from 0. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6136 Test Plan: Find an environment where the test fails, and see it passes with the fix applied. Differential Revision: D18889224 fbshipit-source-id: 7946f3bfea81d236f7bb1554076696705b211b92
-
由 Ziyue Yang 提交于
Summary: BlockBasedTableBuilder uses ExtractUserKey in EnterUnbuffered. This would cause index filter building error, since user-provided timestamp is supported by ExtractUserKeyAndStripTimestamp, and it's used in Add. This commit changes ExtractUserKey to ExtractUserKeyAndStripTimestamp. A test case is also added by modifying DBBasicTestWithTimestampWithParam_ PutAndGet test in db_basic_test to cover ExtractUserKeyAndStripTimestamp usage in both kBuffered and kUnbuffered state of BlockBasedTableBuilder. Before the ExtractUserKeyAndStripTimstamp fix: ``` $ ./db_basic_test --gtest_filter="*PutAndGet*" Note: Google Test filter = *PutAndGet* [==========] Running 2 tests from 1 test case. [----------] Global test environment set-up. [----------] 2 tests from Timestamp/DBBasicTestWithTimestampWithParam [ RUN ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/0 db/db_basic_test.cc:2109: Failure db_->Get(ropts, cfh, "key" + std::to_string(j), &value) NotFound: db/db_basic_test.cc:2109: Failure db_->Get(ropts, cfh, "key" + std::to_string(j), &value) NotFound: db/db_basic_test.cc:2109: Failure db_->Get(ropts, cfh, "key" + std::to_string(j), &value) NotFound: db/db_basic_test.cc:2109: Failure db_->Get(ropts, cfh, "key" + std::to_string(j), &value) NotFound: db/db_basic_test.cc:2109: Failure db_->Get(ropts, cfh, "key" + std::to_string(j), &value) NotFound: [ FAILED ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/0, where GetParam() = false (1177 ms) [ RUN ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/1 [ OK ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/1 (1056 ms) [----------] 2 tests from Timestamp/DBBasicTestWithTimestampWithParam (2233 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test case ran. (2233 ms total) [ PASSED ] 1 test. [ FAILED ] 1 test, listed below: [ FAILED ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/0, where GetParam() = false 1 FAILED TEST ``` After the ExtractUserKeyAndStripTimstamp fix: ``` $ ./db_basic_test --gtest_filter="*PutAndGet*" Note: Google Test filter = *PutAndGet* [==========] Running 2 tests from 1 test case. [----------] Global test environment set-up. [----------] 2 tests from Timestamp/DBBasicTestWithTimestampWithParam [ RUN ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/0 [ OK ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/0 (1417 ms) [ RUN ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/1 [ OK ] Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/1 (1041 ms) [----------] 2 tests from Timestamp/DBBasicTestWithTimestampWithParam (2458 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test case ran. (2458 ms total) [ PASSED ] 2 tests. ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6100 Differential Revision: D18769654 Pulled By: riversand963 fbshipit-source-id: 76c2cf2c9a5e0d85db95d98e812e6af0c2a15c6b
-
由 sdong 提交于
Summary: ASAN reports: internal_repo_rocksdb/repo:db_test - MultiThreaded/MultiThreadedDBTest.MultiThreaded/43: fatal ==2692739==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6130000500ca at pc 0x0000006be780 bp 0x7efef85ccd20 sp 0x7efef85cc4d0 [CONTEXT] === How to use this, how to get the raw stack trace, and more: fburl.com/ASAN === [CONTEXT] READ of size 331 at 0x6130000500ca thread T195 [CONTEXT] #0 db_test_bin+0x6be77f __interceptor_strlen.part.35 [CONTEXT] https://github.com/facebook/rocksdb/issues/1 internal_repo_rocksdb/repo/include/rocksdb/slice.h:55 rocksdb::Slice::Slice(char const*) [CONTEXT] https://github.com/facebook/rocksdb/issues/2 internal_repo_rocksdb/repo/env/io_posix.cc:522 rocksdb::PosixRandomAccessFile::MultiRead(rocksdb::ReadRequest*, unsigned long) I looked at env/io_posix.cc:522 but don't see a reason why the line needs to be there at all, because it is not used before overwritten. So it must be a line that is put there as a bug. Remove it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6135 Test Plan: Rerun the same test which passes after the fix. Run all the tests and make sure they all pass. Differential Revision: D18880251 fbshipit-source-id: 3b84ac6a05b67b529c4202e0ceb4c047460f44f2
-
由 Peter Dillinger 提交于
Summary: Test DBTestUniversalCompaction.RecalculateScoreAfterPicking was flaky on ARM, so it now uses SpecialSkipListFactory (like other tests) for predictable memtable flushes. Fixes https://github.com/facebook/rocksdb/issues/5736 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6125 Test Plan: while ./db_universal_compaction_test; do :; done # for a while on ARM and on Intel (both Linux) Differential Revision: D18864821 Pulled By: pdillinger fbshipit-source-id: 2f3ca0ea66ce420dcd6d41b0ec12377112a5a79f
-
- 09 12月, 2019 2 次提交
-
-
由 sdong 提交于
Summary: db_stress_tool.cc now is a giant file. In order to main it easier to improve and maintain, break it down to multiple source files. Most classes are turned into their own files. Separate .h and .cc files are created for gflag definiations. Another .h and .cc files are created for some common functions. Some test execution logic that is only loosely related to class StressTest is moved to db_stress_driver.h and db_stress_driver.cc. All the files are located under db_stress_tool/. The directory name is created as such because if we end it with either stress or test, .gitignore will ignore any file under it and makes it prone to issues in developements. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6134 Test Plan: Build under GCC7 with and without LITE on using GNU Make. Build with GCC 4.8. Build with cmake with -DWITH_TOOL=1 Differential Revision: D18876064 fbshipit-source-id: b25d0a7451840f31ac0f5ebb0068785f783fdf7d
-
由 suzanwen 提交于
Summary: Isolate `db_bench` from building tests, out of respect for the related comments. Let building tests yields to `WITH_TEST=ON` AND `CMAKE_BUILD_TYPE=Debug` both, and building `db_bench` yields to `WITH_BENCHMARK_TOOLS=ON`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6098 Test Plan: cmake -DCMAKE_BUILD_TYPE=Debug/Release -DWITH_TESTS=ON/OFF -DWITH_BENCHMARK_TOOLS=ON/OFF -DWITH_TOOLS=ON/OFF && make Differential Revision: D18856891 Pulled By: riversand963 fbshipit-source-id: addbee8ad6abefb877843a313b4630cfab3ce4f0
-
- 08 12月, 2019 1 次提交
-
-
由 sdong 提交于
Summary: Right now, PosixRandomAccessFile::MultiRead() executes read requests in parallel. In this PR, it leverages I/O Uring library to run it in parallel, even when page cache is enabled. This function will fall back if the kernel version doesn't support it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5881 Test Plan: Run the unit test on a kernel version supporting it and make sure all tests pass, and run a unit test on kernel version supporting it and see it pass. Before merging, will also run stress test and see it passes. Differential Revision: D17742266 fbshipit-source-id: e05699c925ac04fdb42379456a4e23e4ebcb803a
-
- 07 12月, 2019 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: A longstanding bug in our C interface can trigger this assertion; see issue https://github.com/facebook/rocksdb/issues/6129. Disabling the assertion for now (for 6.6.0) and will re-enable on fix of that bug. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6128 Differential Revision: D18854899 Pulled By: pdillinger fbshipit-source-id: 9eb5294b9f11b208dc1a8cc148aaa31e47ff892b
-
- 06 12月, 2019 3 次提交
-
-
由 Peter Dillinger 提交于
Summary: This test is crashing on ARM but is not yet production code. Let's not let it block ARM CI. See PR https://github.com/facebook/rocksdb/issues/5932 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6126 Test Plan: ./folly_synchronization_distributed_mutex_test, on Linux/ARM, on Linux/x86_64, and with LITE=1 on Linux/x86_64 (also disabled) Differential Revision: D18836576 Pulled By: pdillinger fbshipit-source-id: d8a36eea2f048e8330411d994435d1c58a15d978
-
由 Adam Simpkins 提交于
Summary: PR https://github.com/facebook/rocksdb/issues/5937 changed the db_stress tool to also require db_stress_tool.cc, and updated the Makefile but not the CMakeLists.txt file. This updates the CMakeLists.txt file so that the CMake build succeeds again. PR https://github.com/facebook/rocksdb/issues/5950 updated the Makefile build to package db_stress_tool.cc into its own librocksdb_stress.a library. I haven't done that here since there didn't really seem to be much benefit: the Makefile-based build does not install this library. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6117 Test Plan: Confirmed the CMake build succeeds on an Ubuntu 18.04 system. Differential Revision: D18835053 Pulled By: riversand963 fbshipit-source-id: 6e2a66834716e73b1eb736d9b7159870defffec5
-
由 Jim Meyering 提交于
Summary: Use `#!/usr/bin/env python2.7` instead. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6124 Test Plan: `J=8 make commit_prereq` Differential Revision: D18834668 Pulled By: ltamasi fbshipit-source-id: cec40266cd5bcae8bf6cbe5a564ae78540deccc4
-
- 04 12月, 2019 2 次提交
-
-
由 Yanqin Jin 提交于
Summary: Before this fix, `make all` will emit full compilation command when building object files in the third-party/folly directory even if default verbosity is 0 (AM_DEFAULT_VERBOSITY). Test Plan (devserver): ``` $make all | tee build.log $make check ``` Check build.log to verify. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6120 Differential Revision: D18795621 Pulled By: riversand963 fbshipit-source-id: 04641a8359cd4fd55034e6e797ed85de29ee2fe2
-
由 Connor 提交于
Summary: ``` In file included from /usr/include/c++/4.8.2/algorithm:62:0, from ./db/merge_context.h:7, from ./db/dbformat.h:16, from ./tools/block_cache_analyzer/block_cache_trace_analyzer.h:12, from tools/block_cache_analyzer/block_cache_trace_analyzer.cc:8: /usr/include/c++/4.8.2/bits/stl_algo.h: In instantiation of ‘_RandomAccessIterator std::__unguarded_partition(_RandomAccessIterator, _RandomAccessIterator, const _Tp&, _Compare) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<std::pair<std::basic_string<char>, long unsigned int>*, std::vector<std::pair<std::basic_string<char>, long unsigned int> > >; _Tp = std::pair<std::basic_string<char>, long unsigned int>; _Compare = rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1]’: /usr/include/c++/4.8.2/bits/stl_algo.h:2296:78: required from ‘_RandomAccessIterator std::__unguarded_partition_pivot(_RandomAccessIterator, _RandomAccessIterator, _Compare) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<std::pair<std::basic_string<char>, long unsigned int>*, std::vector<std::pair<std::basic_string<char>, long unsigned int> > >; _Compare = rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1]’ /usr/include/c++/4.8.2/bits/stl_algo.h:2337:62: required from ‘void std::__introsort_loop(_RandomAccessIterator, _RandomAccessIterator, _Size, _Compare) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<std::pair<std::basic_string<char>, long unsigned int>*, std::vector<std::pair<std::basic_string<char>, long unsigned int> > >; _Size = long int; _Compare = rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1]’ /usr/include/c++/4.8.2/bits/stl_algo.h:5499:44: required from ‘void std::sort(_RAIter, _RAIter, _Compare) [with _RAIter = __gnu_cxx::__normal_iterator<std::pair<std::basic_string<char>, long unsigned int>*, std::vector<std::pair<std::basic_string<char>, long unsigned int> > >; _Compare = rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1’ tools/block_cache_analyzer/block_cache_trace_analyzer.cc:583:79: required from here /usr/include/c++/4.8.2/bits/stl_algo.h:2263:35: error: no match for call to ‘(rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1) (std::pair<std::basic_string<char>, long unsigned int>&, const std::pair<std::basic_string<char>, long unsigned int>&)’ while (__comp(*__first, __pivot)) ^ tools/block_cache_analyzer/block_cache_trace_analyzer.cc:582:9: note: candidates are: [=](std::pair<std::string, uint64_t>& a, ^ In file included from /usr/include/c++/4.8.2/algorithm:62:0, from ./db/merge_context.h:7, from ./db/dbformat.h:16, from ./tools/block_cache_analyzer/block_cache_trace_analyzer.h:12, from tools/block_cache_analyzer/block_cache_trace_analyzer.cc:8: /usr/include/c++/4.8.2/bits/stl_algo.h:2263:35: note: bool (*)(std::pair<std::basic_string<char>, long unsigned int>&, std::pair<std::basic_string<char>, long unsigned int>&) <conversion> while (__comp(*__first, __pivot)) ^ /usr/include/c++/4.8.2/bits/stl_algo.h:2263:35: note: candidate expects 3 arguments, 3 provided tools/block_cache_analyzer/block_cache_trace_analyzer.cc:583:46: note: rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1 std::pair<std::string, uint64_t>& b) { return b.second < a.second; }); ^ tools/block_cache_analyzer/block_cache_trace_analyzer.cc:583:46: note: no known conversion for argument 2 from ‘const std::pair<std::basic_string<char>, long unsigned int>’ to ‘std::pair<std::basic_string<char>, long unsigned int>&’ In file included from /usr/include/c++/4.8.2/algorithm:62:0, from ./db/merge_context.h:7, from ./db/dbformat.h:16, from ./tools/block_cache_analyzer/block_cache_trace_analyzer.h:12, from tools/block_cache_analyzer/block_cache_trace_analyzer.cc:8: /usr/include/c++/4.8.2/bits/stl_algo.h:2266:34: error: no match for call to ‘(rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1) (const std::pair<std::basic_string<char>, long unsigned int>&, std::pair<std::basic_string<char>, long unsigned int>&)’ while (__comp(__pivot, *__last)) ^ tools/block_cache_analyzer/block_cache_trace_analyzer.cc:582:9: note: candidates are: [=](std::pair<std::string, uint64_t>& a, ^ In file included from /usr/include/c++/4.8.2/algorithm:62:0, from ./db/merge_context.h:7, from ./db/dbformat.h:16, from ./tools/block_cache_analyzer/block_cache_trace_analyzer.h:12, from tools/block_cache_analyzer/block_cache_trace_analyzer.cc:8: /usr/include/c++/4.8.2/bits/stl_algo.h:2266:34: note: bool (*)(std::pair<std::basic_string<char>, long unsigned int>&, std::pair<std::basic_string<char>, long unsigned int>&) <conversion> while (__comp(__pivot, *__last)) ^ /usr/include/c++/4.8.2/bits/stl_algo.h:2266:34: note: candidate expects 3 arguments, 3 provided tools/block_cache_analyzer/block_cache_trace_analyzer.cc:583:46: note: rocksdb::BlockCacheTraceAnalyzer::WriteSkewness(const string&, const std::vector<long unsigned int>&, rocksdb::TraceType) const::__lambda1 std::pair<std::string, uint64_t>& b) { return b.second < a.second; }); ^ tools/block_cache_analyzer/block_cache_trace_analyzer.cc:583:46: note: no known conversion for argument 1 from ‘const std::pair<std::basic_string<char>, long unsigned int>’ to ‘std::pair<std::basic_string<char>, long unsigned int>&’ ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6106 Differential Revision: D18783943 Pulled By: riversand963 fbshipit-source-id: cc7fc10565f0210b9eebf46b95cb4950ec0b15fa
-
- 03 12月, 2019 3 次提交
-
-
由 Yanqin Jin 提交于
Summary: After secondary instance replays the logs from primary, certain files become obsolete. The secondary should find these files, evict their table readers from table cache and close them. If this is not done, the secondary will hold on to these files and prevent their space from being freed. Test plan (devserver): ``` $./db_secondary_test --gtest_filter=DBSecondaryTest.SecondaryCloseFiles $make check $./db_stress -ops_per_thread=100000 -enable_secondary=true -threads=32 -secondary_catch_up_one_in=10000 -clear_column_family_one_in=1000 -reopen=100 ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6114 Differential Revision: D18769998 Pulled By: riversand963 fbshipit-source-id: 5d1f151567247196164e1b79d8402fa2045b9120
-
由 anand76 提交于
Summary: Inserting an entry in the block cache with 0 length key is a valid use case. Remove the assertion in ```LRUHandle::CalcTotalCharge```. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6115 Differential Revision: D18769693 Pulled By: anand1976 fbshipit-source-id: 34cc159650300dda6d7273480640478f28392cda
-
由 David Palm 提交于
Summary: Adds two missing functions to the C-API: - `rocksdb_block_based_options_set_data_block_index_type` - `rocksdb_block_based_options_set_data_block_hash_ratio` This enables users in other languages to enjoy the new(-ish) feature. The changes here are partially overlapping with [another PR](https://github.com/facebook/rocksdb/pull/5630) but are more focused on the DataBlock indexing options. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6101 Differential Revision: D18765639 fbshipit-source-id: 4a8947e71b179f26fa1eb83c267dd47ee64ac3b3
-
- 28 11月, 2019 6 次提交
-
-
由 Peter Dillinger 提交于
Summary: To reflect changes in PR https://github.com/facebook/rocksdb/issues/6072 This comment also implies that a seemingly valid use-case for max_valid_backups_to_open is flawed: even if you only want to add a new backup without trying to delete, you might need to clean up after a backup creation that never finished. To clean up properly requires opening all backups to get proper ref counts on shared files. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6105 Test Plan: code comment only Differential Revision: D18736716 Pulled By: pdillinger fbshipit-source-id: 2447c0000eefe3a4ca606926bfe922a8456b0cb7
-
由 Yanqin Jin 提交于
Summary: RocksDB should decrement the counter `unscheduled_flushes_` as soon as the bg thread is scheduled. Before this fix, the counter is decremented only when the bg thread starts and picks an element from the flush queue. This may cause more than necessary bg threads to be scheduled. Not a correctness issue, but may affect flush thread count. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6104 Test Plan: ``` make check ``` Differential Revision: D18735584 Pulled By: riversand963 fbshipit-source-id: d36272d4a08a494aeeab6200a3cff7a3d1a2dc10
-
由 Peter Dillinger 提交于
Summary: format_version=5 enables new Bloom filter. Using 2/5 probability for "latest and greatest" rather than naive 1/4. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6102 Test Plan: start 'make blackbox_crash_test' Differential Revision: D18735685 Pulled By: pdillinger fbshipit-source-id: e81529c8a3f53560d246086ee5f92ee7d79a2eab
-
由 Adam Retter 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6094 Differential Revision: D18731951 Pulled By: ltamasi fbshipit-source-id: 5b73c6009c748f6a2a48d4d880b1259980d801d4
-
由 Yingchun Lai 提交于
Summary: fix a typo in struct ReadOptions Pull Request resolved: https://github.com/facebook/rocksdb/pull/6099 Differential Revision: D18729618 fbshipit-source-id: 850a9df71f7c0abebea17feab77b8d5874e8ba0a
-
由 Peter Dillinger 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6097 Differential Revision: D18729661 Pulled By: pdillinger fbshipit-source-id: d2e4a9d6803aad8dd61ececd5c2b861e6f2da73b
-
- 27 11月, 2019 11 次提交
-
-
由 John Ericson 提交于
Summary: From the reset of the code, it looks this this maybe can be unconditionally given the attribute? But I couldn't test with MSVC so I defensively put under CPP. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6075 Differential Revision: D18723749 fbshipit-source-id: 45fc8732c28dd29aab1644225d68f3c6f39bd69b
-
由 sdong 提交于
Summary: options.periodic_compaction_seconds isn't supported when options.max_open_files != -1. It's because that the information of file creation time is stored in table properties and are not guaranteed to be loaded unless options.max_open_files = -1. Relax this constraint by storing the information in manifest. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6090 Test Plan: Pass all existing tests; Modify an existing test to force the manifest value to take 0 to simulate backward compatibility case; manually open the DB generated with the change by release 4.2. Differential Revision: D18702268 fbshipit-source-id: 13e0bd94f546498a04f3dc5fc0d9dff5125ec9eb
-
由 anand76 提交于
Summary: Some of the entries were incorrectly listed under 6.5.0. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6096 Differential Revision: D18722801 Pulled By: gfosco fbshipit-source-id: 18d1187deb6a9d69a8feb68b727d2f720a65f2bc
-
由 Peter Dillinger 提交于
Summary: This change enables custom implementations of FilterPolicy to wrap a variety of NewBloomFilterPolicy and select among them based on contextual information such as table level and compaction style. * Moves FilterBuildingContext to public API and elaborates it with more useful data. (It would be nice to put more general options-like data, but at the time this object is constructed, we are using internal APIs ImmutableCFOptions and MutableCFOptions and don't have easy access to ColumnFamilyOptions that I can tell.) * Renames BloomFilterPolicy::GetFilterBitsBuilderInternal to GetBuilderWithContext, because it's now public. * Plumbs through the table's "level_at_creation" for filter building context. * Simplified some tests by adding GetBuilder() to MockBlockBasedTableTester. * Adds test as DBBloomFilterTest.ContextCustomFilterPolicy, including sample wrapper class LevelAndStyleCustomFilterPolicy. * Fixes a cross-test bug in DBBloomFilterTest.OptimizeFiltersForHits where it does not reset perf context. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6088 Test Plan: make check, valgrind on db_bloom_filter_test Differential Revision: D18697817 Pulled By: pdillinger fbshipit-source-id: 5f987a2d7b07cc7a33670bc08ca6b4ca698c1cf4
-
由 Adam Retter 提交于
Summary: **NOTE**: this also needs to be back-ported to 6.4.6 and possibly older branches if further releases from them is envisaged. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6081 Differential Revision: D18710107 Pulled By: zhichao-cao fbshipit-source-id: 03260f9316566e2bfc12c7d702d6338bb7941e01
-
由 Patrick Double 提交于
Summary: Add the jni library for musl-libc, specifically for incorporating into Alpine based docker images. The classifier is `musl64`. I have signed the CLA electronically. Pull Request resolved: https://github.com/facebook/rocksdb/pull/3143 Differential Revision: D18719372 fbshipit-source-id: 6189d149310b6436d6def7d808566b0234b23313
-
由 Levi Tamasi 提交于
Summary: This patch factors out the logic that reads a (potentially compressed) blob from a file into a separate helper method `GetRawBlobFromFile`, and cleans up the code a bit. Also, errors during decompression are now logged/propagated to the user by returning a `Status` code of `Corruption`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6093 Test Plan: `make check` Differential Revision: D18716673 Pulled By: ltamasi fbshipit-source-id: 44144bc064cab616862d5643f34384f2bae6eb78
-
由 Peter Dillinger 提交于
Summary: There's no technological impediment to allowing the Bloom filter bits/key to be non-integer (fractional/decimal) values, and it provides finer control over the memory vs. accuracy trade-off. This is especially handy in using the format_version=5 Bloom filter in place of the old one, because bits_per_key=9.55 provides the same accuracy as the old bits_per_key=10. This change not only requires refining the logic for choosing the best num_probes for a given bits/key setting, it revealed a flaw in that logic. As bits/key gets higher, the best num_probes for a cache-local Bloom filter is closer to bpk / 2 than to bpk * 0.69, the best choice for a standard Bloom filter. For example, at 16 bits per key, the best num_probes is 9 (FP rate = 0.0843%) not 11 (FP rate = 0.0884%). This change fixes and refines that logic (for the format_version=5 Bloom filter only, just in case) based on empirical tests to find accuracy inflection points between each num_probes. Although bits_per_key is now specified as a double, the new Bloom filter converts/rounds this to "millibits / key" for predictable/precise internal computations. Just in case of unforeseen compatibility issues, we round to the nearest whole number bits / key for the legacy Bloom filter, so as not to unlock new behaviors for it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6092 Test Plan: unit tests included Differential Revision: D18711313 Pulled By: pdillinger fbshipit-source-id: 1aa73295f152a995328cb846ef9157ae8a05522a
-
由 Levi Tamasi 提交于
Summary: The patch refactors and cleans up the logic around creating new blob files by moving the common code of `SelectBlobFile` and `SelectBlobFileTTL` to a new helper method `CreateBlobFileAndWriter`, bringing the implementation of `SelectBlobFile` and `SelectBlobFileTTL` into sync, and increasing encapsulation by adding new constructors for `BlobFile` and `BlobLogHeader`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6066 Test Plan: Ran `make check` and used the BlobDB mode of `db_bench` to sanity test both the TTL and the non-TTL code paths. Differential Revision: D18646921 Pulled By: ltamasi fbshipit-source-id: e5705a84807932e31dccab4f49b3e64369cea26d
-
由 John Ericson 提交于
Summary: This fixes MinGW cross compilation from case-sensative file systems, at no harm to MinGW builds on Windows. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6076 Differential Revision: D18710554 fbshipit-source-id: a9f299ac3aa019f7dbc07ed0c4a79e19cf99b488
-
由 Adam Retter 提交于
Summary: **NOTE**: This also needs to be back-ported to be 6.4.6 Fix a regression introduced in f2bf0b2d by https://github.com/facebook/rocksdb/pull/5674 whereby the compiled library would get the wrong name on PPC64LE platforms. On PPC64LE, the regression caused the library to be named `librocksdbjni-linux64.so` instead of `librocksdbjni-linux-ppc64le.so`. This PR corrects the name back to `librocksdbjni-linux-ppc64le.so` and also corrects the ordering of conditional arguments in the Makefile to match the expected order as defined in the documentation for Make. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6080 Differential Revision: D18710351 fbshipit-source-id: d4db87ef378263b57de7f9edce1b7d15644cf9de
-