- 12 11月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch adds OptionsUtil::LoadOptionsFromFile() and OptionsUtil::LoadLatestOptionsFromDB(), which allow developers to construct DBOptions and ColumnFamilyOptions from a RocksDB options file. Note that most pointer-typed options such as merge_operator will not be constructed. With this API, developers no longer need to remember all the options in order to reopen an existing rocksdb instance like the following: DBOptions db_options; std::vector<std::string> cf_names; std::vector<ColumnFamilyOptions> cf_opts; // Load primitive-typed options from an existing DB OptionsUtil::LoadLatestOptionsFromDB( dbname, &db_options, &cf_names, &cf_opts); // Initialize necessary pointer-typed options cf_opts[0].merge_operator.reset(new MyMergeOperator()); ... // Construct the vector of ColumnFamilyDescriptor std::vector<ColumnFamilyDescriptor> cf_descs; for (size_t i = 0; i < cf_opts.size(); ++i) { cf_descs.emplace_back(cf_names[i], cf_opts[i]); } // Open the DB DB* db = nullptr; std::vector<ColumnFamilyHandle*> cf_handles; auto s = DB::Open(db_options, dbname, cf_descs, &handles, &db); Test Plan: Augment existing tests in column_family_test options_test db_test Reviewers: igor, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49095
-
- 11 11月, 2015 8 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed build failure of RocksDBLite test Test Plan: options_file_test Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50595
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch allows rocksdb to persist options into a file on DB::Open, SetOptions, and Create / Drop ColumnFamily. Options files are created under the same directory as the rocksdb instance. In addition, this patch also adds a fail_if_missing_options_file in DBOptions that makes any function call return non-ok status when it is not able to persist options properly. // If true, then DB::Open / CreateColumnFamily / DropColumnFamily // / SetOptions will fail if options file is not detected or properly // persisted. // // DEFAULT: false bool fail_if_missing_options_file; Options file names are formatted as OPTIONS-<number>, and RocksDB will always keep the latest two options files. Test Plan: Add options_file_test. options_test column_family_test Reviewers: igor, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48285
-
由 Siying Dong 提交于
Make use of portable `uint64_t` type to make possible 64-bit file access
-
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
由 Nathan Bronson 提交于
Summary: Parallel writes will only be possible for certain combinations of flags and WriteBatch contents. Traversing the WriteBatch at write time to check these conditions would be expensive, but it is very cheap to keep track of when building WriteBatch-es. When loading WriteBatch-es during recovery, a deferred computation state is used so that the flags never need to be computed. Test Plan: 1. add asserts and EXPECT_EQ-s 2. make check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50337
-
由 Nathan Bronson 提交于
Summary: Scoped anonymous enums seem to be better supported than static constexpr at the moment, so this diff replaces the latter with the former. Also, this diff removes an incorrect inclusion of pthread.h. MSVC build was broken starting with D50439. Test Plan: 1. build 2. observe proper skiplist behavior by absence of pathological slowdown 3. push diff to tmp_try_windows branch to tickle AppVeyor 4. wait for contbuild before committing to master Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50517
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
- 10 11月, 2015 3 次提交
-
-
由 Nathan Bronson 提交于
Summary: Using a TLS random instance for skiplist makes it smaller (useful for hash_skiplist_rep) and prepares skiplist for concurrent adds. This diff also modifies the branching factor math to avoid an unnecessary division. This diff has the effect of changing the sequence of skip list node height choices made by tests, so it has the potential to cause unit test failures for tests that implicitly rely on the exact structure of the skip list. Tests that try to exactly trigger a compaction are likely suspects for this problem (these tests have always been brittle to changes in the skiplist details). I've minimizes this risk by reseeding the main thread's Random at the beginning of each test, increasing the universal compaction size_ratio limit from 101% to 105% for some tests, and verifying that the tests pass many times. Test Plan: for i in `seq 0 9`; do make check; done Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50439
-
由 Islam AbdelRahman 提交于
Enable C4200, C4702, C4305 windows warnings
-
由 Nathan Bronson 提交于
Summary: Reverting c745f1d2 because it was based on an incorrect understanding of the correct way to enable TSAN tests (it assumes "make COMPILE_WITH_TSAN=1 check" but in fact only "COMPILE_WITH_TSAN=1 make check" is supported). Test Plan: COMPILE_WITH_TSAN=1 make check Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50445
-
- 09 11月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Correct the comment of GetApproximateMemoryUsageByType. Test Plan: No code change. Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50409
-
- 07 11月, 2015 9 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Reverting https://reviews.facebook.net/D34269 for now after I landed it a flaky test started continuously failing, I am almost sure this patch is not related to the test but I will revert it until I figure out why it's failing Test Plan: make check Reviewers: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50385
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
nonstandard extension used : zero-sized array in struct/union
-
由 Islam AbdelRahman 提交于
Enable Windows warnings C4307 C4309 C4512 C4701
-
由 Dmitri Smirnov 提交于
Enable C4307 'operator' : integral constant overflow Longs and ints on Windows are 32-bit hence the overflow Enable C4309 'conversion' : truncation of constant value Enable C4512 'class' : assignment operator could not be generated Enable C4701 Potentially uninitialized local variable 'name' used
-
由 Islam AbdelRahman 提交于
Summary: This diff introduce new table properties that will be written for block based tables These properties are - comparator name - merge operator name - property collectors names Test Plan: - Added a new unit test to verify that these tests are written/read correctly - Running all other tests right now (wont land until all tests finish) Reviewers: rven, kradhakrishnan, igor, sdong, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34269
-
由 Nathan Bronson 提交于
Summary: When a write batch can't join a batch group due to the total size of the contained batches, the write controller's GetDelay is passed a size value that includes the rejected batch. Test Plan: make check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50343
-
由 Nathan Bronson 提交于
Summary: TSAN builds for gcc 4.9 need a PIC version of the libraries taken from the fbcode platform. This is accomplished by assuming every .a has a _pic.a sibling, and by fixing the third-party2 zlib build. Test Plan: make COMPILE_WITH_TSAN=1 check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50331
-
- 06 11月, 2015 7 次提交
-
-
由 agiardullo 提交于
Summary: Docuemented what is currently supported by SingleDelete based on its current implementation. Test Plan: n/a Reviewers: sdong, kradhakrishnan, yhchiang, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50205
-
由 Islam AbdelRahman 提交于
Improve concurrency when running tests
-
由 Venkatesh Radhakrishnan 提交于
Summary: Use IterKey to store prefix_start_ so that it doesn't get freed Test Plan: PrefixTest.PrefixValid Reviewers: anthony, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50289
-
由 Dmitri Smirnov 提交于
PowerShell seems to have a hard time when a flood of async tasks is scheduled at the same time. I speculated that WaitForMultipleObjects() in Windows can only take up to 64 process handles and if you want to handle more than you should write some additional code which can be sub-optimal. I.e to implement Wait-Job -Any. I decided to test that suggestion and introduced a $Concurrency parameter with a default value of 62. So in the new version the script fires up up to $Concurrency value and wait for anything to complete before starting any more processes. This improved matters greatly. Individual tests against ramdrive now run in 8 minutes and all of the 200+ db_tests run in 9 minutes with concurrency values of 8-16. About 48 is required to load a CPU on my box running against HD but that does not improve running times much. Other changes include respect -EnableJE for the individual test exes. Enforce exclusions for the individual tests.
-
由 Satnam Singh 提交于
Summary: Valgrind reports an issue with the test for GeoIterator. This diff explicitly deletes the two iterators used in this test. Test Plan: This diff is for a test. The test still passes. Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50193
-
由 Venkatesh Radhakrishnan 提交于
Summary: MyRocks testing found an issue that while iterating over keys that are outside the prefix, sometimes wrong results were seen for keys outside the prefix. We now tighten the range of keys seen with a new read option called prefix_seen_at_start. This remembers the starting prefix and then compares it on a Next for equality of prefix. If they are from a different prefix, it sets valid to false. Test Plan: PrefixTest.PrefixValid Reviewers: IslamAbdelRahman, sdong, yhchiang, anthony Reviewed By: anthony Subscribers: spetrunia, hermanlee4, yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50211
-
由 Igor Canadi 提交于
Summary: As title. Let's run it for 1 hour. Test Plan: How can I test legocastle changes? Reviewers: IslamAbdelRahman, yhchiang, rven, sdong, anthony, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49653
-
- 05 11月, 2015 5 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move some of the items in public API changes section to to new features section. Test Plan: no code change. Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50229
-
由 Islam AbdelRahman 提交于
Summary: Fix build for clang Test Plan: USE_CLANG=1 make all -j64 make clean make check -j64 Reviewers: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50217
-
由 Islam AbdelRahman 提交于
Fix appveyor build failure
-
由 SherlockNoMad 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch introduces OptionsSanityCheckLevel internally to enable sanity check rocksdb options. Utilities API will be added in the follow-up diffs. Test Plan: Added more tests in options_test Reviewers: igor, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49515
-
- 04 11月, 2015 5 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compile error in RocksDBLite: 18:00:33 CC utilities/memory/memory_test.o 18:00:33 utilities/memory/memory_test.cc: In function ‘int main(int, char**)’: 18:00:33 utilities/memory/memory_test.cc:268:66: error: ‘printf’ was not declared in this scope 18:00:33 printf("Skipped in RocksDBLite as utilities are not supported."); 18:00:33 ^ Test Plan: make OPT=-DROCKSDB_LITE memory_test Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50145
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch introduces utilities/memory, which currently includes GetApproximateMemoryUsageByType that reports different types of rocksdb memory usage given a list of input DBs. The API also take care of the case where Cache could be shared across multiple column families / multiple db instances. Currently, it reports memory usage of memtable, table-readers and cache. Test Plan: utilities/memory/memory_test.cc Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49257
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch adds GetAggregatedIntProperty() that returns the aggregated value from all CFs Test Plan: Added a test in db_test Reviewers: igor, sdong, anthony, IslamAbdelRahman, rven Reviewed By: rven Subscribers: rven, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49497
-
由 Satnam Singh 提交于
-
由 Satnam Singh 提交于
Summary: This diff is a first step towards an iterator based interface for the SearchRadial method which replaces a vector of GeoObjects with an iterator for GeoObjects. This diff works by just wrapping the iterator for the encapsulated vector of GeoObjects. A future diff could extend this approach by defining an interator in terms of the underlying iteration in SearchRadial which would then remove the need to have an in-memory representation for all the matching GeoObjects. Fixes T8421387 Test Plan: The existing tests have been modified to work with the new interface. Reviewers: IslamAbdelRahman, kradhakrishnan, dhruba, igor Reviewed By: igor Subscribers: igor, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50031
-
- 03 11月, 2015 1 次提交
-
-
由 Islam AbdelRahman 提交于
Add Option to Skip Flushing in TableBuilder
-