- 03 6月, 2015 8 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix the following compile warning in db/db_impl db/db_impl.cc:1603:19: error: implicit conversion loses integer precision: 'const uint64_t' (aka 'const unsigned long') to 'int' [-Werror,-Wshorten-64-to-32] info.job_id = job_id; ~ ^~~~~~ Test Plan: db_test Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39423
-
由 Yueh-Hsuan Chiang 提交于
Summary: Allow EventListener::OnCompactionCompleted to return CompactionJobStats, which contains useful information about a compaction. Example CompactionJobStats returned by OnCompactionCompleted(): smallest_output_key_prefix 05000000 largest_output_key_prefix 06990000 elapsed_time 42419 num_input_records 300 num_input_files 3 num_input_files_at_output_level 2 num_output_records 200 num_output_files 1 actual_bytes_input 167200 actual_bytes_output 110688 total_input_raw_key_bytes 5400 total_input_raw_value_bytes 300000 num_records_replaced 100 is_manual_compaction 1 Test Plan: Developed a mega test in db_test which covers 20 variables in CompactionJobStats. Reviewers: rven, igor, anthony, sdong Reviewed By: sdong Subscribers: tnovak, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38463
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed heap-use-after-free error in compaction_job_test.cc Test Plan: compaction_job_test Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39411
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed ROCKSDB_LITE compile error due to the missing of TableFileCreationInfo Test Plan: make OPT=-DROCKSDB_LITE shared_lib -j32 Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39405
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compile warning in listener_test.cc: db/listener_test.cc:214:8: error: 'OnTableFileCreated' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override] 14:16:46 void OnTableFileCreated( Test Plan: make listener_test Reviewers: sdong, igor Subscribers: leveldb
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add EventListener::OnTableFileCreated(), which will be called when a table file is created. This patch is part of the EventLogger and EventListener integration. Test Plan: Augment existing test in db/listener_test.cc Reviewers: anthony, kradhakrishnan, rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38865
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add a stats counter for DB_WRITE back which was mistakenly removed. Test Plan: augment GroupCommitTest Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39399
-
由 sdong 提交于
Summary: In previous change https://reviews.facebook.net/D39099 , while renaming parameters, use a wrong parameter, causing CompactRange() to compact not wrong level. Test Plan: Run "DBTest.MigrateToDynamicLevelMaxBytesBase" which failed with the patch. Reviewers: rven, yhchiang, kradhakrishnan, igor, anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39393
-
- 02 6月, 2015 5 次提交
-
-
由 Mike Kolupaev 提交于
Summary: We occasionally get write stalls (>1s Write() calls) on HDD under read load. The following timers explain almost all of the stalls: - perf_context.db_mutex_lock_nanos - perf_context.db_condition_wait_nanos - iostats_context.open_time - iostats_context.allocate_time - iostats_context.write_time - iostats_context.range_sync_time - iostats_context.logger_time In my experiments each of these occasionally takes >1s on write path under some workload. There are rare cases when Write() takes long but none of these takes long. Test Plan: Added code to our application to write the listed timings to log for slow writes. They usually add up to almost exactly the time Write() call took. Reviewers: rven, yhchiang, sdong Reviewed By: sdong Subscribers: march, dhruba, tnovak Differential Revision: https://reviews.facebook.net/D39177
-
由 sdong 提交于
Summary: In DB::CompactRange(), change parameter "reduce_level" to "change_level". Users can compact all data to the last level if needed. By doing it, users can migrate the DB to options.level_compaction_dynamic_level_bytes=true. Test Plan: Add a unit test for it. Reviewers: yhchiang, anthony, kradhakrishnan, igor, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39099
-
由 Yueh-Hsuan Chiang 提交于
Summary: DBImpl::notifying_events_ is a internal counter in DBImpl which is used to prevent DB close when DB is notifying events. However, as the current events all rely on either compaction or flush which already have similar counters to prevent DB close, it is safe to remove notifying_events_. Test Plan: listener_test examples/compact_files_example Reviewers: igor, anthony, kradhakrishnan, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39315
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed compile warning in compact_files_example.cc Test Plan: compact_files_example Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39309
-
由 Mike Kolupaev 提交于
Summary: It used to be no good (known to me) non-intrusive way to wrap WritableFile - you can't call protected virtual methods of the wrapped pointer to WritableFile. This diff adds a convenience class WritableFileWrapper that makes wrapping WritableFile both possible and easy. Test Plan: `make clean; make -j release`, `make clean; OPT=-DROCKSDB_LITE make release`, `make clean; USE_CLANG=1 make -j all`. Reviewers: sdong, yhchiang, rven Reviewed By: rven Subscribers: dhruba, tnovak, march Differential Revision: https://reviews.facebook.net/D39147
-
- 01 6月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
WriteBatch.Merge w/ SliceParts support
-
- 31 5月, 2015 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed db_stress by correcting the verification of column family names in the Listener of db_stress Test Plan: db_stress Reviewers: igor, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39255
-
由 Igor Canadi 提交于
Summary: As title Test Plan: make check Reviewers: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39243
-
- 30 5月, 2015 7 次提交
-
-
由 agiardullo 提交于
Summary: Broken by optimistic transaction diff. (I only built 'release' not 'static_lib' when testing). Test Plan: build Reviewers: yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39219
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a compile warning in db_stress in NDEBUG mode. Test Plan: make OPT=-DNDEBUG db_stress Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39213
-
由 agiardullo 提交于
Summary: Optimistic transactions supporting begin/commit/rollback semantics. Currently relies on checking the memtable to determine if there are any collisions at commit time. Not yet implemented would be a way of enuring the memtable has some minimum amount of history so that we won't fail to commit when the memtable is empty. You should probably start with transaction.h to get an overview of what is currently supported. Test Plan: Added a new test, but still need to look into stress testing. Reviewers: yhchiang, igor, rven, sdong Reviewed By: sdong Subscribers: adamretter, MarkCallaghan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D33435
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compile warning in db_stress: error: 'OnCompactionCompleted' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override] Test Plan: make db_stress Reviewers: sdong, igor, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39207
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a compile error in ROCKSDB_LITE Test Plan: make db_stress OPT=-DROCKSDB_LITE -j32 Reviewers: sdong, igor, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39201
-
由 Yueh-Hsuan Chiang 提交于
Summary: Include EventListener in stress test. Test Plan: make blackbox_crash_test whitebox_crash_test Reviewers: anthony, igor, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39105
-
由 Igor Canadi 提交于
Summary: as title Test Plan: make release Reviewers: MarkCallaghan, sdong Reviewed By: sdong Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38853
-
- 29 5月, 2015 4 次提交
-
-
由 Reed Allman 提交于
also hooked up WriteBatchInternal
-
由 agiardullo 提交于
Summary: For transactions, we are using the memtables to validate that there are no write conflicts. But after flushing, we don't have any memtables, and transactions could fail to commit. So we want to someone keep around some extra history to use for conflict checking. In addition, we want to provide a way to increase the size of this history if too many transactions fail to commit. After chatting with people, it seems like everyone prefers just using Memtables to store this history (instead of a separate history structure). It seems like the best place for this is abstracted inside the memtable_list. I decide to create a separate list in MemtableListVersion as using the same list complicated the flush/installalflushresults logic too much. This diff adds a new parameter to control how much memtable history to keep around after flushing. However, it sounds like people aren't too fond of adding new parameters. So I am making the default size of flushed+not-flushed memtables be set to max_write_buffers. This should not change the maximum amount of memory used, but make it more likely we're using closer the the limit. (We are now postponing deleting flushed memtables until the max_write_buffer limit is reached). So while we might use more memory on average, we are still obeying the limit set (and you could argue it's better to go ahead and use up memory now instead of waiting for a write stall to happen to test this limit). However, if people are opposed to this default behavior, we can easily set it to 0 and require this parameter be set in order to use transactions. Test Plan: Added a xfunc test to play around with setting different values of this parameter in all tests. Added testing in memtablelist_test and planning on adding more testing here. Reviewers: sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37443
-
由 Yueh-Hsuan Chiang 提交于
Summary: Rename EventLoggerHelpers EventHelpers, as it's going to include all event-related helper functions instead of EventLogger only stuffs. Test Plan: make Reviewers: sdong, rven, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39093
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move listeners from ColumnFamilyOptions to DBOptions Test Plan: listener_test compact_files_test Reviewers: rven, anthony, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39087
-
- 27 5月, 2015 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Compaction now boosts the size of deletion entries of a file only when the number of deletion entries is greater than the number of non-deletion entries in the file. The motivation here is that in a stable workload, the number of deletion entries should be roughly equal to the number of non-deletion entries. If we compensate the size of deletion entries in a stable workload, the deletion compensation logic might introduce unwanted effet which changes the shape of LSM tree. Test Plan: db_test --gtest_filter="*Deletion*" Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38703
-
由 Igor Canadi 提交于
C: add more block based table stuff, some aux slice transform/merge ops
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a missing "}" at the end of the generated JSON Log in EventLoggerHelpers::LogTableFileCreation. Test Plan: db_bench Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38919
-
由 Yueh-Hsuan Chiang 提交于
Summary: Removed an unused private variable in db_impl.h Test Plan: make db_test Reviewers: sdong, anthony, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38925
-
- 23 5月, 2015 9 次提交
-
-
由 Reed Allman 提交于
-
由 Reed Allman 提交于
-
由 Igor Canadi 提交于
adding docker build script and dockerfile for tools
-
由 agiardullo 提交于
Summary: fix typo Test Plan: none Reviewers: tfarina, igor Reviewed By: tfarina, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37347
-
由 Aaron Schlesinger 提交于
-
由 Aaron Schlesinger 提交于
-
由 Igor Canadi 提交于
Summary: We have a bug where we don't report the last level's files as being compacted. This fixes it. Test Plan: See the fix in action here: https://phabricator.fb.com/P19845738 Reviewers: MarkCallaghan, sdong Reviewed By: sdong Subscribers: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38727
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch fixes the following two bugs on logging file deletion. 1. Previously, file deletion failure was only logged in INFO_LEVEL. This patch changes it to ERROR_LEVEL and does some code clean. 2. EventLogger previously will always generate the same log on table file deletion even when file deletion is not successful. Now the resulting status of file deletion will also be logged. Test Plan: make all check Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38817
-
由 Yueh-Hsuan Chiang 提交于
Summary: Change the log-level of DB summary and options from INFO_LEVEL to WARN_LEVEL Test Plan: Use db_bench to verify the log level. Sample output: 2015/05/22-00:20:39.778064 7fff75b41300 [WARN] RocksDB version: 3.11.0 2015/05/22-00:20:39.778095 7fff75b41300 [WARN] Git sha rocksdb_build_git_sha:7fee8775 2015/05/22-00:20:39.778099 7fff75b41300 [WARN] Compile date May 22 2015 2015/05/22-00:20:39.778101 7fff75b41300 [WARN] DB SUMMARY 2015/05/22-00:20:39.778145 7fff75b41300 [WARN] SST files in /tmp/rocksdbtest-691931916/dbbench dir, Total Num: 0, files: 2015/05/22-00:20:39.778148 7fff75b41300 [WARN] Write Ahead Log file in /tmp/rocksdbtest-691931916/dbbench: 2015/05/22-00:20:39.778150 7fff75b41300 [WARN] Options.error_if_exists: 0 2015/05/22-00:20:39.778152 7fff75b41300 [WARN] Options.create_if_missing: 1 2015/05/22-00:20:39.778153 7fff75b41300 [WARN] Options.paranoid_checks: 1 Reviewers: MarkCallaghan, igor, kradhakrishnan Reviewed By: igor Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38835
-