- 17 12月, 2014 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Avoid unnecessary unlock and lock mutex when notifying events. Test Plan: ./listener_test Reviewers: igor Reviewed By: igor Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30267
-
由 Venkatesh Radhakrishnan 提交于
Summary: We now release the mutex before copying the files in the case of the trivial move. This path does not use the compaction job. Test Plan: DBTest.LevelCompactionThirdPath Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30381
-
- 16 12月, 2014 1 次提交
-
-
由 Venkatesh Radhakrishnan 提交于
Summary: Allow Level-style compaction to place files in different paths This diff provides the code for task 4854591. We now support level-compaction to place files in different paths by specifying them in db_paths along with the minimum level for files to store in that path. Test Plan: ManualLevelCompactionOutputPathId in db_test.cc Reviewers: yhchiang, MarkCallaghan, dhruba, yoshinorim, sdong Reviewed By: sdong Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29799
-
- 13 12月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: ASAN build fails once for this error: 14:04:52 ==== Test DBTest.CompactFilesOnLevelCompaction 14:04:52 db_test: db/version_set.cc:1062: void rocksdb::VersionStorageInfo::AddFile(int, rocksdb::FileMetaData*): Assertion `level <= 0 || level_files->empty() || internal_comparator_->Compare( (*level_files)[level_files->size() - 1]->largest, f->smallest) < 0' failed. Not abling figure out reason. We use std:vector for sorting for save and add one more assert to help figure out whether it is the sorting's problem. Test Plan: make all check Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D30117
-
- 12 12月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: Now DB::GetSnapshot() doesn't scale to more column families, as it needs to go through all the column families to find whether snapshot is supported. This patch optimizes it. Test Plan: Add unit tests to cover negative cases. make all check Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30093
-
- 11 12月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: Set operations in VerisonBuilder is shown as a performance bottleneck of restarting DB when there are lots of files. Make both of added_files and deleted_files use unordered set or map. Only when adding the files, sort the added files. Test Plan: make all check Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: hermanlee4, leveldb, dhruba, ljin Differential Revision: https://reviews.facebook.net/D30051
-
- 09 12月, 2014 2 次提交
-
-
由 sdong 提交于
Summary: This is a regression bug introduced by https://reviews.facebook.net/D24729 . max_total_wal_size would be off the target it should be more and more in the case that the a user holds the current super version after flush or compaction. This patch fixes it Test Plan: make all check Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: ljin, yoshinorim, MarkCallaghan, hermanlee4, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29961
-
由 Leonidas Galanis 提交于
Summary: When wal_dir is used, DestroyDB is not passed the wal_dir option and so we get a Corruption exception. Test Plan: Verified manually that the following command line works now: ./db_bench --db=/mnt/db/rocksdb ... --disable_wal=0 --wal_dir=/data/users/rocksdb/WAL... --benchmarks=filluniquerandom --use_existing_db=0... Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29859
-
- 06 12月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: Add a counter in SnapshotList to show number of snapshots. Also a unix timestamp in every snapshot. Add two DB Properties to return number of snapshots and timestamp of the oldest one. Test Plan: Add unit test checking Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba, MarkCallaghan Differential Revision: https://reviews.facebook.net/D29919
-
- 05 12月, 2014 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Remove the use of exception in WriteBatch::Handler. Now the default implementations of Put, Merge, and Delete in WriteBatch::Handler are no-op. Test Plan: Add three test cases in write_batch_test ./write_batch_test Reviewers: sdong, igor Reviewed By: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29835
-
由 Yueh-Hsuan Chiang 提交于
Summary: Replace exception by setting valid_ = false in DBIter::MergeValuesNewToOld(). Test Plan: Not sure if I am right at this, but it seems we currently don't have a good way to test that code path as it requires dynamically set merge_operator = nullptr at the time while Merge() is calling. Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29811
-
- 04 12月, 2014 1 次提交
-
-
由 Mark Callaghan 提交于
Summary: Adds counter for bytes moved (files pushed down a level rather than compacted) to compaction IO stats as Moved(GB). From the output removed these infrequently used columns: RW-Amp, Rn(cnt), Rnp1(cnt), Wnp1(cnt), Wnew(cnt). Example old output: Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s) Rn(cnt) Rnp1(cnt) Wnp1(cnt) Wnew(cnt) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms) RecordIn RecordDrop ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 0/0 0 0.0 0.0 0.0 0.0 2130.8 2130.8 0.0 0.0 0.0 109.1 0 0 0 0 20002 25068 0.798 28.75 182059 0.16 0 0 L1 142/0 509 1.0 4618.5 2036.5 2582.0 4602.1 2020.2 4.5 2.3 88.5 88.1 24220 701246 1215528 514282 53466 4229 12.643 0.00 0 0.002032745988 300688729 Example new output: Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms) RecordIn RecordDrop -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- L0 7/0 13 1.8 0.0 0.0 0.0 0.6 0.6 0.0 0.0 0.0 14.7 44 353 0.124 0.03 626 0.05 0 0 L1 9/0 16 1.6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0 0 0.000 0.00 0 0.00 0 0 Task ID: # Blame Rev: Test Plan: make check, run db_bench --fillseq --stats_per_interval --stats_interval and look at output Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D29787
-
- 03 12月, 2014 2 次提交
-
-
由 Jonah Cohen 提交于
Summary: Introduces a new class for managing write buffer memory across column families. We supplement ColumnFamilyOptions::write_buffer_size with ColumnFamilyOptions::write_buffer, a shared pointer to a WriteBuffer instance that enforces memory limits before flushing out to disk. Test Plan: Added SharedWriteBuffer unit test to db_test.cc Reviewers: sdong, rven, ljin, igor Reviewed By: igor Subscribers: tnovak, yhchiang, dhruba, xjin, MarkCallaghan, yoshinorim Differential Revision: https://reviews.facebook.net/D22581
-
由 Igor Canadi 提交于
Summary: Two fixes: 1. if cpplint is not present on the system, don't return a confusing error in the linter 2. Add include_alpha, which means our includes should be sorted lexicographically Test Plan: Tried unsorting our includes, lint complained Reviewers: rven, ljin, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28845
-
- 27 11月, 2014 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Block Universal and FIFO compactions in ROCKSDB_LITE Test Plan: make shared_lib -j32 make OPT=-DROCKSDB_LITE shared_lib Reviewers: ljin, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29589
-
由 Yueh-Hsuan Chiang 提交于
Summary: db_imp_readonly.o is one of the big obj file. If it's not a necessary feature, we should probably block it in ROCKSDB_LITE. 1322704 Nov 24 16:55 db/db_impl_readonly.o Test Plan: make shared_lib -j32 make ROCKSDB_LITE shared_lib -j32 Reviewers: ljin, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29583
-
- 26 11月, 2014 3 次提交
-
-
由 Matt Amos 提交于
Store upper bound `Slice` with the same lifetime as the `ReadOptions` so that we can provide a pointer to it.
-
由 Matt Amos 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Block internal_stats in ROCKSDB_LITE. Test Plan: make OPT=-DROCKSDB_LITE shared_lib Reviewers: ljin, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29541
-
- 25 11月, 2014 6 次提交
-
-
由 Reed Allman 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: In some environment such as android, the c++ library does not have std::to_string. This path adds rocksdb::ToString(), which wraps std::to_string when std::to_string is not available, and implements std::to_string in the other case. Test Plan: make dbg -j32 ./db_test make clean make dbg OPT=-DOS_ANDROID -j32 ./db_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29181
-
由 Yueh-Hsuan Chiang 提交于
Summary: Improve listener_test to avoid possible false alarm Test Plan: ./listener_test
-
由 Lei Jin 提交于
Summary: This compaction trigger does not seem to test any thing specific to cuckoo table. Remove it. Test Plan: make all check Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29523
-
由 Lei Jin 提交于
Summary: Reported by bootcamper This causes ldb tool to fail the assertion in ~ColumnFamilyData() Test Plan: ./ldb --db=/tmp/test_db1 --create_if_missing put a1 b1 ./ldb manifest_dump --path=/tmp/test_db1/MANIFEST-000001 Reviewers: sdong, yhchiang, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29517
-
由 Venkatesh Radhakrishnan 提交于
Summary: Free checkpoint after its directory is removed. Test Plan: Run valgrind with GetSnapshotLink. Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29493
-
- 22 11月, 2014 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: An entry of ConstantColumnFamilyInfo is created when: 1. DB::Open 2. CreateColumnFamily. However, there are cases that DB::Open could also call CreateColumnFamily when create_missing_column_families=true. As a result, it will create duplicate ConstantColumnFamilyInfo and one of them would be leaked. Test Plan: ./deletefile_test Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29307
-
由 sdong 提交于
Summary: "build all" breaks in Clang mode with db_bench. Fix it. Test Plan: USE_CLANG=1 make all Reviewers: ljin, rven, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D29379
-
由 Yueh-Hsuan Chiang 提交于
Summary: Improve listener_test by ensuring flushes are completed before assert. Test Plan: listener_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29319
-
由 Igor Canadi 提交于
Summary: stringSplit is not how we name our functions. Also, we had two StringSplit's in the codebase Test Plan: make check Reviewers: yhchiang, dhruba Reviewed By: dhruba Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29361
-
- 21 11月, 2014 3 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add enable_thread_tracking to DBOptions to allow tracking thread status related to the DB. Default is off. Test Plan: export ROCKSDB_TESTS=ThreadList ./db_test Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29289
-
由 Venkatesh Radhakrishnan 提交于
Summary: Moved checkpoint to utilities. Addressed comments by Igor, Siying, Dhruba Test Plan: db_test/SnapshotLink Reviewers: dhruba, igor, sdong Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29079
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add GetThreadList API, which allows developer to track the status of each process. Currently, calling GetThreadList will only get the list of background threads in RocksDB with their thread-id and thread-type (priority) set. Will add more support on this in the later diffs. ThreadStatus currently has the following properties: // An unique ID for the thread. const uint64_t thread_id; // The type of the thread, it could be ROCKSDB_HIGH_PRIORITY, // ROCKSDB_LOW_PRIORITY, and USER_THREAD const ThreadType thread_type; // The name of the DB instance where the thread is currently // involved with. It would be set to empty string if the thread // does not involve in any DB operation. const std::string db_name; // The name of the column family where the thread is currently // It would be set to empty string if the thread does not involve // in any column family. const std::string cf_name; // The event that the current thread is involved. // It would be set to empty string if the information about event // is not currently available. Test Plan: ./thread_list_test export ROCKSDB_TESTS=GetThreadList ./db_test Reviewers: rven, igor, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D25047
-
- 19 11月, 2014 2 次提交
-
-
由 Lei Jin 提交于
Summary: The very last reference happens in DBImpl::GetOptions() I built with both DBImpl::GetOptions() and ColumnFamilyData::options() commented out Test Plan: make all check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29073
-
由 Lei Jin 提交于
Summary: We should not reference cfd->options() directly! Test Plan: make release Reviewers: sdong, rven, igor, yhchiang Reviewed By: igor, yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29061
-
- 17 11月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a bug which could hide non-ok status in CompactionJob::Run() Test Plan: make Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28995
-
- 15 11月, 2014 5 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: This way we can gurantee that old MemTables get destructed before DBImpl gets destructed, which might be useful if we want to make them depend on state from DBImpl. Test Plan: make check with asserts in JobContext's destructor Reviewers: ljin, sdong, yhchiang, rven, jonahcohen Reviewed By: jonahcohen Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28959
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-