- 09 12月, 2015 1 次提交
-
- 08 12月, 2015 6 次提交
-
-
由 agiardullo 提交于
Summary: D50475 enables using SST files for transaction write-conflict checking. In order for this to work, we need to make sure not to compact out SingleDeletes when there is an earlier transaction snapshot(D50295). If there is a long-held snapshot, this could reduce the benefit of the SingleDelete optimization. This diff allows Transactions to mark snapshots as being used for write-conflict checking. Then, during compaction, we will be able to optimize SingleDeletes better in the future. This diff adds a flag to SnapshotImpl which is used by Transactions. This diff also passes the earliest write-conflict snapshot's sequence number to CompactionIterator. This diff does not actually change Compaction (after this diff is pushed, D50295 will be able to use this information). Test Plan: no behavior change, ran existing tests Reviewers: rven, kradhakrishnan, yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51183
-
由 sdong 提交于
Summary: DBTest.DynamicCompactionOptions ocasionally fails during valgrind run. We sent a sleeping task to block compaction thread pool but we don't wait it to run. Test Plan: Run the test multiple times in an environment which can cause failure. Reviewers: rven, kradhakrishnan, igor, IslamAbdelRahman, anthony, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51687
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch fix a race condition in persisting options which will cause a crash when: * Thread A obtain cf options and start to persist options based on that cf options. * Thread B kicks in and finish DropColumnFamily and delete cf_handle. * Thread A wakes up and tries to finish the persisting options and crashes. Test Plan: Add a test in column_family_test that can reproduce the crash Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51609
-
由 Venkatesh Radhakrishnan 提交于
Summary: Several tests in db_compaction_test are failing with aborts in valgrind. These are LevelCompactionThirdPath, LevelCompactionPathUse and CompressLevelCompaction. We now use the SpecialSkipListFactory to make them more deterministic Test Plan: valgrind Reviewers: anthony, yhchiang, kradhakrishnan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51663
-
由 sdong 提交于
Summary: After the skip list optimization, ColumnFamilyTest.DifferentWriteBufferSizes can occasionally fail with flush triggering of column family 3. Insert more data to it to make sure flush will trigger. Test Plan: Run it multiple times with both of jemaloc on and off and see it always passes. (Without thd commit the run with jemalloc fails with chance of about one in two) Reviewers: rven, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51645
-
- 05 12月, 2015 3 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: db_universal_compaction_test is still failing because of UniversalCompactionNumLevels/DBTestUniversalCompaction.UniversalCompactionSecondPathRatio/0 https://travis-ci.org/facebook/rocksdb/jobs/94949919 Use same approach to fix other tests to fix this test Test Plan: Run ./db_universal_compaction_test on mac and make sure all the tests pass Reviewers: kradhakrishnan, yhchiang, rven, anthony, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51591
-
由 krad 提交于
Summary: Skip list now cannot estimate memory across allocators consistently and hence triggers flush at different time. This breaks certain unit tests. The fix is to adopt key count instead of size for flush. Test Plan: Ran test on dev box and mac (where it used to fail) Reviewers: sdong CC: leveldb@ Task ID: #9273334 Blame Rev:
-
由 Jay Edgar 提交于
Summary: When SetSnapshot() is used the caller immediately knows a snapshot has been created, but when SetSnapshotOnNextOperation() is used the caller needs a way to get notified when that snapshot has been generated. This creates an interface that the client can implement that will be called at the time the snapshot is created. Test Plan: Added a new SetSnapshotOnNextOperationWithNotification test into the transaction_test. Reviewers: sdong, anthony Reviewed By: anthony Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51177
-
- 04 12月, 2015 2 次提交
-
-
由 Alex Yang 提交于
Summary: Fixes T8781168. Added a new function EnableAutoCompactions in db.h to be publicly avialable. This allows compaction to be re-enabled after disabling it via SetOptions Refactored code to set the dbptr earlier on in TransactionDB::Open and DB::Open Temporarily disable auto_compaction in TransactionDB::Open until dbptr is set to prevent race condition. Test Plan: Ran make all check verified fix on myrocks side: was able to reproduce the seg fault with ../tools/mysqltest.sh --mem --force rocksdb.drop_table method was to manually sleep the thread after DB::Open but before TransactionDB ptr was assigned in transaction_db_impl.cc: DB::Open(db_options, dbname, column_families_copy, handles, &db); clock_t goal = (60000 * 10) + clock(); while (goal > clock()); ...dbptr(aka rdb) gets assigned below verified my changes fixed the issue. Also added unit test 'ToggleAutoCompaction' in transaction_test.cc Reviewers: hermanlee4, anthony Reviewed By: anthony Subscribers: alex, dhruba Differential Revision: https://reviews.facebook.net/D51147
-
由 Islam AbdelRahman 提交于
Avoid empty ranges vector with subsequent zero element access
-
- 03 12月, 2015 3 次提交
-
-
由 yuslepukhin 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix incorrect merge in db/db_compaction_test.cc Test Plan: db_compaction_test Reviewers: igor, sdong, anthony, IslamAbdelRahman, rven, kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51531
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make DBCompactionTestWithParam::CompactionTrigger more deterministic Test Plan: ./db_compaction_test Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51507
-
- 02 12月, 2015 3 次提交
-
-
由 sdong 提交于
Summary: Verifiction condition of DBTest.SuggestCompactRangeTest is too strict. Based on key distribution, we might have more small files in last level. Not check number of files in the last level. Test Plan: Run DBTest.SuggestCompactRangeTest with both of jemalloc on and off. Reviewers: rven, IslamAbdelRahman, yhchiang, kradhakrishnan, igor, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51501
-
由 sdong 提交于
Summary: DBTest.DynamicCompactionOptions sometimes fails the assert but I can't repro it locally. Make it more deterministic and readable and see whether the problem is still there. Test Plan: Run tht test and make sure it passes Reviewers: kradhakrishnan, yhchiang, igor, rven, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51309
-
由 sdong 提交于
Summary: DBCompactionTestWithParam.CompactionTrigger fails in non-jemalloc build, after the skip list memtable change. Fix it by making mem table flush trigger by number of entries. Test Plan: Run the test using both of jemalloc and non-jemalloc build. Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, igor, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51471
-
- 01 12月, 2015 9 次提交
-
-
由 sdong 提交于
Summary: With recent commit 33e0c938, db iterator skips perf context counter internal_key_skipped_count when blindly issuing internal Next(). Now increment the counter by one when issuing this Next() Test Plan: Run all existing tests Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan, igor, anthony Reviewed By: anthony Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51465
-
由 agiardullo 提交于
Summary: fix clang build Test Plan: build Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51453
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed some typos in the comments of rocksdb options file example Test Plan: No code change. Reviewers: igor, anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51393
-
由 sdong 提交于
Summary: DBTest.SuggestCompactRangeTest fails for the case when jemalloc is disabled, including ASAN and valgrind builds. It is caused by the improvement of skip list, which allocates different size of nodes for a new records. Fix it by using a special mem table that triggers a flush by number of entries. In that way the behavior will be consistent for all allocators. Test Plan: Run the test with both of DISABLE_JEMALLOC=1 and 0 Reviewers: anthony, rven, yhchiang, kradhakrishnan, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51423
-
由 sdong 提交于
DB to only flush the column family with the largest memtable while option.db_write_buffer_size is hit Summary: When option.db_write_buffer_size is hit, we currently flush all column families. Move to flush the column family with the largest active memt table instead. In this way, we can avoid too many small files in some cases. Test Plan: Modify test DBTest.SharedWriteBuffer to work with the updated behavior Reviewers: kradhakrishnan, yhchiang, rven, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: march, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51291
-
由 Siying Dong 提交于
Fix ms version Appveyor build error
-
由 SherlockNoMad 提交于
-
由 sdong 提交于
Summary: options.row_cache should already been initialized as null by default. Still try to set it following current convention, because one valgrind failure reports a failure related to it. Test Plan: Run all unit tests Reviewers: yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51303
-
由 Islam AbdelRahman 提交于
Summary: Fix clang Test Plan: make check Reviewers: sdong, yhchiang, rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51417
-
- 26 11月, 2015 3 次提交
-
-
由 sdong 提交于
Summary: Getting file size from all the backup files can take a long time. In some cases, the sizes are available in file names. We allow a mode to get those sizes from file name. Test Plan: Make some unit tests in backupable_db_test to run in such a mode. Make sure RocksDB Lite builds too. Reviewers: IslamAbdelRahman, rven, yhchiang, kradhakrishnan, anthony, igor Reviewed By: igor Subscribers: muthu, asameet, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51243
-
由 Igor Canadi 提交于
EnvWrapper: add ReuseWritableFile
-
由 Sage Weil 提交于
This was missed when ReuseWritableFile was added to Env in 1bcafb62. Signed-off-by: NSage Weil <sage@redhat.com>
-
- 25 11月, 2015 6 次提交
-
-
由 sdong 提交于
Summary: Now DBIter::Next() always compares with current key with itself first, which is unnecessary if the last key is not a merge key. I made the change and didn't see db_iter_test fails. Want to hear whether people have any idea what I miss. Test Plan: Run all unit tests Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48279
-
由 Nathan Bronson 提交于
Summary: This diff completes the creation of InlineSkipList<Cmp>, which is like SkipList<const char*, Cmp> but it always allocates the key contiguously with the node. This allows us to remove the pointer from the node to the key. As a result the memory usage of the skip list is reduced (by 1 to sizeof(void*) bytes depending on the padding required to align the key storage), cache locality is improved, and we halve the number of calls to the allocator. For skip lists whose keys are freshly-allocated const char*, InlineSkipList is stricly preferrable to SkipList. This diff doesn't replace SkipList, however, because some of the use cases of SkipList in RocksDB are either character sequences that are not allocated at the same time as the skip list node allocation (for example hash_linklist_rep) or have different key types (for example write_batch_with_index). Taking advantage of inline allocation for those cases is left to future work. The perf win is biggest for small values. For single-threaded CPU-bound (32M fillrandom operations with no WAL log) with 16 byte keys and 0 byte values, the db_bench perf goes from ~310k ops/sec to ~410k ops/sec. For large values the improvement is less pronounced, but seems to be between 5% and 10% on the same configuration. Test Plan: make check Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51123
-
由 Nathan Bronson 提交于
Summary: This diff is 2/3 in a sequence that introduces a skip list optimized for a key that is a freshly-allocated const char*. The change is broken into pieces to make it easier to review. This piece removes the Key template type, introduces the AllocateKey interface, and changes the unit test from using uint64_t as the Key type to using pointers to an 8 byte blob. Test Plan: unit test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51285
-
由 Nathan Bronson 提交于
Summary: This diff is 1/3 in a sequence that introduces a skip list optimized for a key that is a freshly-allocated const char*. The diff is broken into pieces to make it easier to review. This piece only introduces the new type by copying the existing SkipList, with mechanical naming changes and reformatting. Test Plan: new unit test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51279
-
由 Igor Canadi 提交于
Enable override to 3rd party linkage
-
由 Dmitri Smirnov 提交于
-
- 21 11月, 2015 4 次提交
-
-
由 agiardullo 提交于
Summary: DBTest.DynamicLevelCompressionPerLevel2 sometimes fails during valgrind runs. This causes our valgrind tests to fail. Not sure what the best fix is for this test, but hopefully this simple change is sufficient. Test Plan: run test Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51111
-
由 Siying Dong 提交于
Build on Visual Studio 2015 Update 1
-
由 Venkatesh Radhakrishnan 提交于
Summary: Provide an API for compaction filter to specify that it needs to be applied even if there are snapshots. Test Plan: DBTestCompactionFilter.CompactionFilterIgnoreSnapshot Reviewers: yhchiang, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51087
-
由 yuslepukhin 提交于
-