- 05 8月, 2015 2 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Introduce DeleteScheduler that allow enforcing a rate limit on file deletion Instead of deleting files immediately, files are moved to trash directory and deleted in a background thread that apply sleep penalty between deletes if needed. I have updated PurgeObsoleteFiles and PurgeObsoleteWALFiles to use the delete_scheduler instead of env_->DeleteFile Test Plan: added delete_scheduler_test existing unit tests Reviewers: kradhakrishnan, anthony, rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43221
-
由 sdong 提交于
Summary: crash_test now only runs complicated options, multiple column families, prefix hash, frequently changing options, many compaction threads, etc. These options are good to cover new features but we loss coverage in most common use cases. Furthermore, by running only for multiple column families, we are not able to create LSM trees that are large enough to cover some stress cases. Make half of crash_test runs the simply tests: single column family, default mem table, one compaction thread, no change options. Test Plan: Run crash_test Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43461
-
- 04 8月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff adds CompactOnDeletionCollector in utilities/table_properties_collectors, which applies a sliding window to a sst file and mark this file as need-compaction when it observe enough deletion entries within the consecutive keys covered by the sliding window. Test Plan: compact_on_deletion_collector_test Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yoshinorim, sdong Reviewed By: sdong Subscribers: maykov, dhruba Differential Revision: https://reviews.facebook.net/D41175
-
- 25 7月, 2015 1 次提交
-
-
由 Andres Noetzli 提交于
Summary: Whenever a Java class implements equals(), it has to implement hashCode(), otherwise there might be weird behavior when inserting instances of the class in a hash map for example. This adds two missing hashCode() implementations and extends tests to test the hashCode() implementations. Test Plan: make jtest Reviewers: rven, igor, sdong, yhchiang Reviewed By: yhchiang Subscribers: anthony, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43017
-
- 23 7月, 2015 1 次提交
-
-
由 Dave Cottlehuber 提交于
- OSX format is libname.major.minor.patch.dylib - closes #666
-
- 22 7月, 2015 1 次提交
-
-
由 Siying Dong 提交于
Summary: From other ones' investigation: "sync_file_range() behavior highly depends on kernel version and filesystem. xfs does neighbor page flushing outside of the specified ranges. For example, sync_file_range(fd, 8192, 16384) does not only trigger flushing page #3 to #4, but also flushing many more dirty pages (i.e. up to page#16)... Ranges of the sync_file_range() should be far enough from write() offset (at least 1MB)." Test Plan: make all check Reviewers: igor, rven, kradhakrishnan, yhchiang, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: yoshinorim, MarkCallaghan, sumeet, domas, dhruba, leveldb, ljin Differential Revision: https://reviews.facebook.net/D15807
-
- 21 7月, 2015 7 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move general compaction tests from db_test.cc to db_compaction_test.cc Test Plan: db_test db_compaction_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42651
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add db_inplace_update_test back to Makefile Test Plan: make all check Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42717
-
由 Yueh-Hsuan Chiang 提交于
Summary: Revert Makefile Test Plan: make clean Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42711
-
由 Yueh-Hsuan Chiang 提交于
Summary: This reverts commit 03467bdd. It seems the previous git merge remove all the tabs in Makefile and cause the Makefile not runnable. This patch revert that incorrect merge. Test Plan: make Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42705 Conflicts: CMakeLists.txt Makefile
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move in-place-update related tests from db_test.cc to db_inplace_update_test.cc Test Plan: db_test db_inplace_update_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42657
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add missing db_log_iter_test in the test list Test Plan: make all check Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42663
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed DBTestUniversalManualCompactionOutputPathId test by changing the expected number of files when setting up the test as flushes no-longer preempt compactions in patch https://reviews.facebook.net/D41931. Also, include db_universal_copaction_test in make all check. Test Plan: db_universal_copaction_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42639
-
- 18 7月, 2015 2 次提交
-
-
由 Igor Canadi 提交于
Summary: When we first started, max_background_flushes was 0 by default and compaction thread was executing flushes (since there was no flush thread). Then, we switched the default max_background_flushes to 1. However, we still support the case where there is no flush thread and flushes are done in compaction. This is making our code a bit more complicated. By not supporting this use-case we can make our code simpler. We have a special case that when you set max_background_flushes to 0, we schedule the flush to execute on the compaction thread. Test Plan: make check (there might be some unit tests that depend on this behavior) Reviewers: IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41931
-
由 Andres Notzli 提交于
Summary: MergeUntil was not reporting a success when merging an operand with a Value/Deletion despite the comments in MergeHelper and CompactionJob indicating otherwise. This lead to operands being written to the compaction output unnecessarily: M1 M2 M3 P M4 M5 --> (P+M1+M2+M3) M2 M3 M4 M5 (before the diff) M1 M2 M3 P M4 M5 --> (P+M1+M2+M3) M4 M5 (after the diff) In addition, the code handling Values/Deletion was basically identical. This patch unifies the code. Finally, this patch also adds testing for merge_helper. Test Plan: make && make check Reviewers: sdong, rven, yhchiang, tnovak, igor Reviewed By: igor Subscribers: tnovak, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42351
-
- 15 7月, 2015 6 次提交
-
-
由 lovro 提交于
Summary: Repeat of b6655a67 (reverted in b7a2369f) with a proper fix for the issue that 57d216ea was trying to fix. Test Plan: make check for i in $(seq 100); do ./db_stress --test_batches_snapshots=1 --threads=32 --write_buffer_size=4194304 --destroy_db_initially=0 --reopen=20 --readpercent=45 --prefixpercent=5 --writepercent=35 --delpercent=5 --iterpercent=10 --db=/tmp/rocksdb_crashtest_KdCI5F --max_key=100000000 --mmap_read=0 --block_size=16384 --cache_size=1048576 --open_files=500000 --verify_checksum=1 --sync=0 --progress_reports=0 --disable_wal=0 --disable_data_sync=1 --target_file_size_base=2097152 --target_file_size_multiplier=2 --max_write_buffer_number=3 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --filter_deletes=0 --memtablerep=prefix_hash --prefix_size=7 --ops_per_thread=200 || break; done Reviewers: anthony, sdong, igor, yhchiang Reviewed By: igor, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D41391
-
由 Igor Canadi 提交于
Summary: For some reason, parallel tests are failing in Jenkins. We turned off parallel tests for all tests except `make coverage`. Let's turn it off here, too, so that we can have Jenkins test working again. For the error log, see: https://ci-builds.fb.com/job/rocksdb_coverage/166/console Test Plan: commit and wait for jenkins Reviewers: meyering, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42057
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move UniversalCompaction related db-tests to db_universal_compaction_test.cc Test Plan: db_test db_universal_compaction_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42225
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move TailingIterator tests from db_test.cc to db_test_tailing_iterator.cc Test Plan: db_test db_test_tailing_iterator Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42021
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make TransactionLogIterator related tests from db_test.cc to db_log_iter_test.cc Test Plan: db_test db_log_iter_test Reviewers: sdong, IslamAbdelRahman, igor, anthony Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42045
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move CompactionFilter tests in db_test.cc to db_compaction_filter_test.cc Test Plan: db_test db_compaction_filter_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42207
-
- 14 7月, 2015 3 次提交
-
-
由 Igor Canadi 提交于
Summary: This is revert of D40449. I'm not sure why our tests failed before, they seem to be working fine on my devbox. Test Plan: ran `make check` Reviewers: anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41955
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move DynamicLevel related db-tests to db_dynamic_level_test.cc Test Plan: db_dynamic_level_test db_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42039
-
由 Yueh-Hsuan Chiang 提交于
Summary: Move reusable part of db_test.cc to util/db_test_util.h. This makes it more possible to partition db_test.cc into multiple smaller test files. Also, fixed many old lint errors in db_test. Test Plan: db_test Reviewers: igor, anthony, IslamAbdelRahman, sdong, kradhakrishnan Reviewed By: sdong, kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41973
-
- 24 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Remove -Wl,--no-as-needed flag when making shared_lib in OSX and IOS as those environment doe not have compile option --no-as-needed ld: unknown option: --no-as-needed clang: error: linker command failed with exit code 1 (use -v to see invocation) Test Plan: make shared_lib Reviewers: meyering, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40353
-
- 20 6月, 2015 3 次提交
-
-
由 Michael Callahan 提交于
Summary: Recent checkin added ldb_test.py to the make check target but the test fails. Remove it again for now and make task. Test Plan: No more ldb_tests.py running Reviewers: igor, anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40449
-
由 Michael Callahan 提交于
Summary: Hack up rocksdb_dump and rocksdb_undump utilities to get this task rolling/promote discussion. Test Plan: Dump/undump databases recursively to see if nothing is lost. Reviewers: sdong, yhchiang, rven, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37269
-
由 Venkatesh Radhakrishnan 提交于
Summary: When there are multiple column families, the flush in GetLiveFiles is not atomic, so that there are entries in the wal files which are needed to get a consisten RocksDB. We now add the log files to the checkpoint. Test Plan: CheckpointCF - This test forces more data to be written to the other column families after the flush of the first column family but before the second. Reviewers: igor, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40323
-
- 19 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Remove compactor_test, which depends on a directory not exist in our code base. make compactor_test GEN util/build_version.cc GEN util/build_version.cc make: *** No rule to make target `utilities/compaction/compactor_test.o', needed by `compactor_test'. Stop. Test Plan: verify the output message of make compactor_test Reviewers: rven, anthony, kradhakrishnan, igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40341
-
- 11 6月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: The motivation is here: http://stackoverflow.com/questions/30706348/rocksdb-make-error Apparently there is a bug (?) in g++ where we don't link rt library properly when building RocksDB's shared library. It looks like this works when we add an option -Wl, --no-as-needed. Test Plan: make shared_lib Reviewers: meyering, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39681
-
- 03 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Allow EventListener::OnCompactionCompleted to return CompactionJobStats, which contains useful information about a compaction. Example CompactionJobStats returned by OnCompactionCompleted(): smallest_output_key_prefix 05000000 largest_output_key_prefix 06990000 elapsed_time 42419 num_input_records 300 num_input_files 3 num_input_files_at_output_level 2 num_output_records 200 num_output_files 1 actual_bytes_input 167200 actual_bytes_output 110688 total_input_raw_key_bytes 5400 total_input_raw_value_bytes 300000 num_records_replaced 100 is_manual_compaction 1 Test Plan: Developed a mega test in db_test which covers 20 variables in CompactionJobStats. Reviewers: rven, igor, anthony, sdong Reviewed By: sdong Subscribers: tnovak, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38463
-
- 30 5月, 2015 2 次提交
-
-
由 agiardullo 提交于
Summary: Optimistic transactions supporting begin/commit/rollback semantics. Currently relies on checking the memtable to determine if there are any collisions at commit time. Not yet implemented would be a way of enuring the memtable has some minimum amount of history so that we won't fail to commit when the memtable is empty. You should probably start with transaction.h to get an overview of what is currently supported. Test Plan: Added a new test, but still need to look into stress testing. Reviewers: yhchiang, igor, rven, sdong Reviewed By: sdong Subscribers: adamretter, MarkCallaghan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D33435
-
由 Igor Canadi 提交于
Summary: as title Test Plan: make release Reviewers: MarkCallaghan, sdong Reviewed By: sdong Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38853
-
- 23 5月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Sometimes we want to run tests sequentially. J=1 gives us that option Test Plan: make J=1 check -- sequential make J=2 check -- parallel Reviewers: sdong, yhchiang, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38805
-
- 20 5月, 2015 2 次提交
-
-
由 DerekSchenk 提交于
Includes the LDFLAGS so that the correct libraries will be linked. This links rt to resolve the issue https://github.com/facebook/rocksdb/issues/606.
-
由 Igor Canadi 提交于
Summary: In third-party2 build we need to force git sha because we're compiling from a different git repositry. Test Plan: `FORCE_GIT_SHA=igor make` Reviewers: kradhakrishnan, sdong Reviewed By: kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38679
-
- 16 5月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: Now rocksdb_build_git_sha is determined from "git sha". It is hard if the release is not from the repository directly but from a source code copy. Change to use the versions given in Makefile. Test Plan: Run "make util/build_version.cc" Reviewers: kradhakrishnan, rven, meyering, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D38451
-
- 09 5月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Don't treat warnings as error when building rocksdbjavastatic Test Plan: make rocksdbjavastatic -j32 Reviewers: rven, fyrz, adamretter, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38187
-
- 08 5月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Without this I get bunch of questions when I run `make clean` Test Plan: no more questions! Reviewers: rven, yhchiang, meyering, anthony Reviewed By: meyering, anthony Subscribers: meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38145
-
- 06 5月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: This will reduce errors reported by open source by 90%. Developers should have warnings break their compile. Users shouldn't. Test Plan: make dbg: g++ -g -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-unused-parameter -Werror -I. -I./include -std=c++11 -DROCKSDB_PLATFORM_POSIX -DOS_LINUX -fno-builtin-memcmp -DROCKSDB_FALLOCATE_PRESENT -DSNAPPY -DGFLAGS=gflags -DZLIB -DBZIP2 -march=native -isystem ./third-party/gtest-1.7.0/fused-src -Woverloaded-virtual -Wn on-virtual-dtor -Wno-missing-field-initializers -c db/repair.cc -o db/repair.o make all: g++ -g -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-unused-parameter -Werror -I. -I./include -std=c++11 -DROCKSDB_PLATFORM_POSIX -DOS_LINUX -fno-builtin-memcmp -DROCKSDB_FALLOCATE_PRESENT -DSNAPPY -DGFLAGS=gflags -DZLIB -DBZIP2 -march=native -isystem ./third-party/gtest-1.7.0/fused-src -O2 -fno-omit-frame-pointer -momit-leaf-frame-pointer -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -c db/repair.cc -o db/repair.o make static_lib: g++ -g -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-unused-parameter -I. -I./include -std=c++11 -DROCKSDB_PLATFORM_POSIX -DOS_LINUX -fno-builtin-memcmp -DROCKSDB_FALLOCATE_PRESENT -DSNAPPY -DGFLAGS=gflags -DZLIB -DBZIP2 -march=native -isystem ./third-party/gtest-1.7.0/fused-src -O2 -fno-omit-frame-pointer -momit-leaf-frame-pointer -DNDEBUG -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -c db/repair.cc -o db/repair.o Reviewers: sdong, yhchiang, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38031
-