- 08 11月, 2014 10 次提交
-
-
由 Igor Canadi 提交于
Summary: Based on @sdong's feedback in the diff, we shouldn't keep db_mutex in CompactionJob's state. This diff removes db_mutex from CompactionJob state, by making next_file_number_ atomic. That way we only need to pass the lock to InstallCompactionResults() because of LogAndApply() Test Plan: make check Reviewers: ljin, yhchiang, rven, sdong Reviewed By: sdong Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28491
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the shadowing in db/compaction.cc and include/rocksdb/db.h Test Plan: make
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed compile error in db/db_impl.cc Test Plan: make
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed compile error in db/flush_job.cc Test Plan: make
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed compile error in db/compaction.cc and db/compaction_picker.cc Test Plan: make
-
由 Igor Canadi 提交于
Summary: Previously I made `make check` work with -Wshadow, but there are some tools that are not compiled using `make check`. Test Plan: make all Reviewers: yhchiang, rven, ljin, sdong Reviewed By: ljin, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28497
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed -WShadow errors in db/db_test.cc and include/rocksdb/metadata.h Test Plan: make
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff adds three sets of APIs to RocksDB. = GetColumnFamilyMetaData = * This APIs allow users to obtain the current state of a RocksDB instance on one column family. * See GetColumnFamilyMetaData in include/rocksdb/db.h = EventListener = * A virtual class that allows users to implement a set of call-back functions which will be called when specific events of a RocksDB instance happens. * To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners = CompactFiles = * CompactFiles API inputs a set of file numbers and an output level, and RocksDB will try to compact those files into the specified level. = Example = * Example code can be found in example/compact_files_example.cc, which implements a simple external compactor using EventListener, GetColumnFamilyMetaData, and CompactFiles API. Test Plan: listener_test compactor_test example/compact_files_example export ROCKSDB_TESTS=CompactFiles db_test export ROCKSDB_TESTS=MetaData db_test Reviewers: ljin, igor, rven, sdong Reviewed By: sdong Subscribers: MarkCallaghan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D24705
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Here's a prototype of redesigning pending_outputs_. This way, we don't have to expose pending_outputs_ to other classes (CompactionJob, FlushJob, MemtableList). DBImpl takes care of it. Still have to write some comments, but should be good enough to start the discussion. Test Plan: make check, will also run stress test Reviewers: ljin, sdong, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28353
-
- 07 11月, 2014 7 次提交
-
-
由 Jonah Cohen 提交于
Summary: CompactionDeletionTriggerReopen wasn't actually testing universal compaction. Test Plan: db_test Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D28443
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make PartialCompactionFailure Test more robust again by blocking background compaction until we simulate the file creation error. Test Plan: export ROCKSDB_TESTS=PartialCompactionFailure ./db_test Reviewers: sdong, igor, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28431
-
由 Lei Jin 提交于
Summary: TEST_WaitForFlush should wait until it sees error when parameter is set to true so we don't need to loop and timeout Test Plan: ROCKSDB_TESTS=DropWritesFlush ./db_test Reviewers: sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28419
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make PartialCompactionFailure Test more robust. Test Plan: export ROCKSDB_TESTS=PartialCompactionFailure ./db_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28425
-
由 Igor Canadi 提交于
Summary: It turns out that -Wshadow has different rules for gcc than clang. Previous commit fixed clang. This commits fixes the rest of the warnings for gcc. Test Plan: compiles Reviewers: ljin, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28131
-
由 sdong 提交于
Summary: In order to avoid random failure of DBTest.GroupCommitTest, artificially sleep 100 microseconds in each log writing. Test Plan: Run the test in a machine where valgrind version of the test always fails multiple times and see it always succeed. Reviewers: igor, yhchiang, rven, ljin Reviewed By: ljin Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28401
-
由 sdong 提交于
Summary: This patch makes it a contract that if an empty write batch is passed to DB::Write() and WriteOptions.sync = true, fsync is called to WAL. Test Plan: A new unit test Reviewers: ljin, rven, yhchiang, igor Reviewed By: igor Subscribers: dhruba, MarkCallaghan, leveldb Differential Revision: https://reviews.facebook.net/D28365
-
- 06 11月, 2014 2 次提交
-
-
由 Shi Feng 提交于
Summary: Add db_bench option --report_file_operations Test Plan: ./db_bench --report_file_operations Observe outputs on # of file operations Reviewers: ljin, MarkCallaghan, sdong Reviewed By: sdong Subscribers: yhchiang, rven, igor, dhruba Differential Revision: https://reviews.facebook.net/D27945
-
由 sdong 提交于
Summary: 1. fix possible overflow of the two stats by using uint64_t 2. use a similar source of data to calculate RecordDrop. Previous one is not correct. Test Plan: See outputs of db_bench settings, and the results look reasonable Reviewers: MarkCallaghan, ljin, igor Reviewed By: igor Subscribers: rven, leveldb, yhchiang, dhruba Differential Revision: https://reviews.facebook.net/D28155
-
- 05 11月, 2014 9 次提交
-
-
由 Lei Jin 提交于
Summary: as title Test Plan: ./db_test Reviewers: sdong, yhchiang, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28269
-
由 Lei Jin 提交于
Summary: as title Test Plan: ran it Reviewers: yhchiang, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28311
-
由 sdong 提交于
Replace some ASSERT_TRUE() asserts in DBTest.DynamicMemtableOptions and DynamicCompactionOptions with more specific ones Summary: Replace some ASSERT_TRUE() to ASSERT_GT() and ASSERT_LT() so that in case the assert is triggered, the value is printed out. Test Plan: Run the two tests Reviewers: ljin, rven, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28293
-
由 sdong 提交于
Summary: Now DBTest.DynamicMemtableOptions sets background compaction to be 4, without actually increasing thread pool size (even before the feature of automatic increasing it). To make sure the behavior stays the same after the automatic thread pool increasing, set it back to 1. Hopefully it can fix the occasional failure of the test. Test Plan: Run the test Reviewers: igor, ljin Reviewed By: ljin Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28281
-
由 Yueh-Hsuan Chiang 提交于
Summary: Apply InfoLogLevel to the logs in db/compaction_job.cc Test Plan: db_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: leveldb, MarkCallaghan, dhruba Differential Revision: https://reviews.facebook.net/D28275
-
由 Yueh-Hsuan Chiang 提交于
Summary: Apply InfoLogLevel to the logs in db/version_set.cc Test Plan: make Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27879
-
由 Yueh-Hsuan Chiang 提交于
Summary: Apply InfoLogLevel to the logs in db/wal_manager.cc Test Plan: db_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28239
-
由 Yueh-Hsuan Chiang 提交于
Summary: Apply InfoLogLevel to the logs in db/db_impl.cc Test Plan: db_test db_bench Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: leveldb, MarkCallaghan, dhruba Differential Revision: https://reviews.facebook.net/D28233
-
由 sdong 提交于
Summary: Enforce the accessier naming convention in functions in version_set.h Test Plan: make all check Reviewers: ljin, yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28143
-
- 04 11月, 2014 1 次提交
-
-
由 sdong 提交于
DB::Open() to automatically increase thread pool size if it is smaller than max number of parallel compactions or flushes Summary: With the patch, thread pool size will be automatically increased if DB's options ask for more parallelism of compactions or flushes. Too many users have been confused by the API. Change it to make it harder for users to make mistakes Test Plan: Add two unit tests to cover the function. Reviewers: yhchiang, rven, igor, MarkCallaghan, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27555
-
- 01 11月, 2014 9 次提交
-
-
由 sdong 提交于
Summary: Move all the logic of VersionBuilder to a separate .cc file Test Plan: make all check Reviewers: ljin, yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28083
-
由 Igor Canadi 提交于
Summary: Long awaited CompactionJob class! Move most compaction-related things from DBImpl to CompactionJob, making CompactionJob easier to test and understand. Currently this is just replicating exactly the same functionality with as little as change as possible. As future work, we should: 1. Add CompactionJob tests (I think I'll do that tomorrow) 2. Reduce CompactionJob's state that it inherits from DBImpl 3. Figure out how to do yielding to flush better. Currently I implemented a callback as we agreed yesterday, but I don't think it's a good long term solution. This reduces db_impl.cc from 5000+ LOC to 3400! Test Plan: make check, will add CompactionJob-specific tests, probably also move some tests from db_test to compaction_job_test Reviewers: rven, yhchiang, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27957
-
由 Lei Jin 提交于
Summary: TestMemEnv simulates all Env APIs using in-memory data structures. We can use it to speed up db_test run, which is now reduced ~7mins when it is enabled. We can also add features to simulate power/disk failures in the next step TestMemEnv is derived from helper/mem_env mem_env can not be used for rocksdb since some of its APIs do not give the same results as env_posix. And its file read/write is not thread safe Test Plan: make all -j32 ./db_test ./env_mem_test Reviewers: sdong, yhchiang, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28035
-
由 Lei Jin 提交于
Summary: as title Test Plan: make release Reviewers: sdong, yhchiang, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28053
-
由 Lei Jin 提交于
Summary: as title Test Plan: ./c_test Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28119
-
由 sdong 提交于
Summary: now we use %8d for RecordIn and %10d for RecordDrop, which is far too small for some use cases. Extend both of them to %12d. Test Plan: run one test in db_test and see the LOG file. Reviewers: igor, MarkCallaghan, ljin Reviewed By: ljin Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28041
-
由 sdong 提交于
Summary: BaseReferencedVersionBuilder now unreference version before destructing VersionBuilder, which is wrong. Fix it. Test Plan: make all check valgrind test to tests that used to fail Reviewers: igor, yhchiang, rven, ljin Reviewed By: ljin Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D28101
-
由 Igor Canadi 提交于
Summary: ...and fix all the errors :) Jim suggested turning on -Wshadow because it helped him fix number of critical bugs in fbcode. I think it's a good idea to be -Wshadow clean. Test Plan: compiles Reviewers: yhchiang, rven, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27711
-
由 sdong 提交于
Summary: Rename Version::Builder to VersionBuilder and expose its definition to a header. Make VerisonBuilder not reference Version or ColumnFamilyData, only working with VersionStorageInfo. Add version_builder_test which has a simple test. Test Plan: make all check Reviewers: rven, yhchiang, igor, ljin Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D27969
-
- 31 10月, 2014 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Apply InfoLogLevel to the logs in db/db_iter.cc Test Plan: make Reviewers: igor, ljin, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27861
-
由 sdong 提交于
Summary: CompactionPickerTest.Level1Trigger2 now depends on the STL implementation to be correct. Fix it. Test Plan: Run the test Reviewers: ljin, yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D27963
-