- 17 11月, 2015 4 次提交
-
-
由 Dmitri Smirnov 提交于
-
由 Islam AbdelRahman 提交于
Summary: ``` arc2 lint --everything ``` run the linter on the whole code repo to fix exisitng lint issues Test Plan: make check -j64 Reviewers: sdong, rven, anthony, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50769
-
由 sdong 提交于
Summary: Currently RocksDB may break in lines like this: for (size_t i = sorted_runs.size() - 1; i >= first_index_after; i--) { if options.level0_file_num_compaction_trigger=0. Fix it by not executing the logic of picking compactions if there is no file (sorted_runs.size() = 0). Also internally set options.level0_file_num_compaction_trigger=1 if users give a 0. 0 is a value makes no sense in RocksDB. Test Plan: Run all tests. Will add a unit test too. Reviewers: yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50727
-
由 Venkatesh Radhakrishnan 提交于
Summary: Fixed Rocksdb lite build failure in forward_iterator_bench by defining main for the ROCKSDB_LITE case Test Plan: build ROCKSDB_LITE Reviewers: anthony, yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50733
-
- 14 11月, 2015 2 次提交
-
-
由 Venkatesh Radhakrishnan 提交于
Summary: Under a tailing workload, there were increased block cache misses when a memtable was flushed because we were rebuilding iterators in that case since the version set changed. This was exacerbated in the case of iterate_upper_bound, since file iterators which were over the iterate_upper_bound would have been deleted and are now brought back as part of the Rebuild, only to be deleted again. We now renew the iterators and only build iterators for files which are added and delete file iterators for files which are deleted. Refer to https://reviews.facebook.net/D50463 for previous version Test Plan: DBTestTailingIterator.TailingIteratorTrimSeekToNext Reviewers: anthony, IslamAbdelRahman, igor, tnovak, yhchiang, sdong Reviewed By: sdong Subscribers: yhchiang, march, dhruba, leveldb, lovro Differential Revision: https://reviews.facebook.net/D50679
-
由 Venkatesh Radhakrishnan 提交于
Summary: Since level 0 files can overlap, two level 0 compactions cannot run in parallel. Compact files needs to check this before running a compaction. Test Plan: CompactFilesTest.L0ConflictsFiles Reviewers: igor, IslamAbdelRahman, anthony, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50079
-
- 13 11月, 2015 7 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add CheckOptionsCompatibility() API to options_util that returns Status::OK if the input DBOptions and ColumnFamilyDescriptors are compatible with the latest options stored in the specified DB path. Test Plan: Added tests in options_util_test Reviewers: igor, anthony, IslamAbdelRahman, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50649
-
由 Siying Dong 提交于
Try running db_test during integration build
-
由 yuslepukhin 提交于
-
由 yuslepukhin 提交于
-
由 yuslepukhin 提交于
Add sequential rerun for any failed tests. Add env_test case. Limit concurrency Allow to specify individual tests Take $Limit into account when displaying number of tests
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed valgrind error in options_util_test by deleting the compaction_filter allocated from RandomInitCFOptions(). Test Plan: valgrind --error-exitcode=2 --leak-check=full ./options_util_test Reviewers: anthony, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50661
-
由 Nathan Bronson 提交于
Summary: There's no need for WriteImpl to flatten the write batch group into a single WriteBatch if the WAL is disabled. This diff moves the flattening into the WAL step, and skips flattening entirely if it isn't needed. It's good for about 5% speedup on a multi-threaded workload with no WAL. This diff also adds clarifying comments about the chance for partial failure of WriteBatchInternal::InsertInto, and always sets bg_error_ if the memtable state diverges from the logged state or if a WriteBatch succeeds only partially. Benchmark for speedup: db_bench -benchmarks=fillrandom -threads=16 -batch_size=1 -memtablerep=skip_list -value_size=0 --num=200000 -level0_slowdown_writes_trigger=9999 -level0_stop_writes_trigger=9999 -disable_auto_compactions --max_write_buffer_number=8 -max_background_flushes=8 --disable_wal --write_buffer_size=160000000 Test Plan: asserts + make check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50583
-
- 12 11月, 2015 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: DBCompactionTest.SkipStatsUpdateTest relies on the number of files opened during the DB::Open process, but the persisting options file support altered this number and thus makes DBCompactionTest.SkipStatsUpdateTest in certain environment. This patch fixed this test failure. Test Plan: db_compaction_test Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50637
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch adds OptionsUtil::LoadOptionsFromFile() and OptionsUtil::LoadLatestOptionsFromDB(), which allow developers to construct DBOptions and ColumnFamilyOptions from a RocksDB options file. Note that most pointer-typed options such as merge_operator will not be constructed. With this API, developers no longer need to remember all the options in order to reopen an existing rocksdb instance like the following: DBOptions db_options; std::vector<std::string> cf_names; std::vector<ColumnFamilyOptions> cf_opts; // Load primitive-typed options from an existing DB OptionsUtil::LoadLatestOptionsFromDB( dbname, &db_options, &cf_names, &cf_opts); // Initialize necessary pointer-typed options cf_opts[0].merge_operator.reset(new MyMergeOperator()); ... // Construct the vector of ColumnFamilyDescriptor std::vector<ColumnFamilyDescriptor> cf_descs; for (size_t i = 0; i < cf_opts.size(); ++i) { cf_descs.emplace_back(cf_names[i], cf_opts[i]); } // Open the DB DB* db = nullptr; std::vector<ColumnFamilyHandle*> cf_handles; auto s = DB::Open(db_options, dbname, cf_descs, &handles, &db); Test Plan: Augment existing tests in column_family_test options_test db_test Reviewers: igor, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49095
-
- 11 11月, 2015 10 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed build failure of RocksDBLite test Test Plan: options_file_test Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50595
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch allows rocksdb to persist options into a file on DB::Open, SetOptions, and Create / Drop ColumnFamily. Options files are created under the same directory as the rocksdb instance. In addition, this patch also adds a fail_if_missing_options_file in DBOptions that makes any function call return non-ok status when it is not able to persist options properly. // If true, then DB::Open / CreateColumnFamily / DropColumnFamily // / SetOptions will fail if options file is not detected or properly // persisted. // // DEFAULT: false bool fail_if_missing_options_file; Options file names are formatted as OPTIONS-<number>, and RocksDB will always keep the latest two options files. Test Plan: Add options_file_test. options_test column_family_test Reviewers: igor, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48285
-
由 Siying Dong 提交于
Make use of portable `uint64_t` type to make possible 64-bit file access
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
-
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
由 Nathan Bronson 提交于
Summary: Parallel writes will only be possible for certain combinations of flags and WriteBatch contents. Traversing the WriteBatch at write time to check these conditions would be expensive, but it is very cheap to keep track of when building WriteBatch-es. When loading WriteBatch-es during recovery, a deferred computation state is used so that the flags never need to be computed. Test Plan: 1. add asserts and EXPECT_EQ-s 2. make check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50337
-
由 Nathan Bronson 提交于
Summary: Scoped anonymous enums seem to be better supported than static constexpr at the moment, so this diff replaces the latter with the former. Also, this diff removes an incorrect inclusion of pthread.h. MSVC build was broken starting with D50439. Test Plan: 1. build 2. observe proper skiplist behavior by absence of pathological slowdown 3. push diff to tmp_try_windows branch to tickle AppVeyor 4. wait for contbuild before committing to master Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50517
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
- 10 11月, 2015 3 次提交
-
-
由 Nathan Bronson 提交于
Summary: Using a TLS random instance for skiplist makes it smaller (useful for hash_skiplist_rep) and prepares skiplist for concurrent adds. This diff also modifies the branching factor math to avoid an unnecessary division. This diff has the effect of changing the sequence of skip list node height choices made by tests, so it has the potential to cause unit test failures for tests that implicitly rely on the exact structure of the skip list. Tests that try to exactly trigger a compaction are likely suspects for this problem (these tests have always been brittle to changes in the skiplist details). I've minimizes this risk by reseeding the main thread's Random at the beginning of each test, increasing the universal compaction size_ratio limit from 101% to 105% for some tests, and verifying that the tests pass many times. Test Plan: for i in `seq 0 9`; do make check; done Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50439
-
由 Islam AbdelRahman 提交于
Enable C4200, C4702, C4305 windows warnings
-
由 Nathan Bronson 提交于
Summary: Reverting c745f1d2 because it was based on an incorrect understanding of the correct way to enable TSAN tests (it assumes "make COMPILE_WITH_TSAN=1 check" but in fact only "COMPILE_WITH_TSAN=1 make check" is supported). Test Plan: COMPILE_WITH_TSAN=1 make check Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50445
-
- 09 11月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Correct the comment of GetApproximateMemoryUsageByType. Test Plan: No code change. Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50409
-
- 07 11月, 2015 9 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Reverting https://reviews.facebook.net/D34269 for now after I landed it a flaky test started continuously failing, I am almost sure this patch is not related to the test but I will revert it until I figure out why it's failing Test Plan: make check Reviewers: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50385
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
nonstandard extension used : zero-sized array in struct/union
-
由 Islam AbdelRahman 提交于
Enable Windows warnings C4307 C4309 C4512 C4701
-
由 Dmitri Smirnov 提交于
Enable C4307 'operator' : integral constant overflow Longs and ints on Windows are 32-bit hence the overflow Enable C4309 'conversion' : truncation of constant value Enable C4512 'class' : assignment operator could not be generated Enable C4701 Potentially uninitialized local variable 'name' used
-
由 Islam AbdelRahman 提交于
Summary: This diff introduce new table properties that will be written for block based tables These properties are - comparator name - merge operator name - property collectors names Test Plan: - Added a new unit test to verify that these tests are written/read correctly - Running all other tests right now (wont land until all tests finish) Reviewers: rven, kradhakrishnan, igor, sdong, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34269
-
由 Nathan Bronson 提交于
Summary: When a write batch can't join a batch group due to the total size of the contained batches, the write controller's GetDelay is passed a size value that includes the rejected batch. Test Plan: make check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50343
-
由 Nathan Bronson 提交于
Summary: TSAN builds for gcc 4.9 need a PIC version of the libraries taken from the fbcode platform. This is accomplished by assuming every .a has a _pic.a sibling, and by fixing the third-party2 zlib build. Test Plan: make COMPILE_WITH_TSAN=1 check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50331
-
- 06 11月, 2015 2 次提交
-
-
由 agiardullo 提交于
Summary: Docuemented what is currently supported by SingleDelete based on its current implementation. Test Plan: n/a Reviewers: sdong, kradhakrishnan, yhchiang, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50205
-
由 Islam AbdelRahman 提交于
Improve concurrency when running tests
-