- 24 4月, 2015 9 次提交
-
-
由 sdong 提交于
Summary: Make it build for CYGWIN. Need to define "-std=gnu++11" instead of "-std=c++11" and use some replacement functions. Test Plan: Build it and run some unit tests in CYGWIN Reviewers: yhchiang, rven, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37605
-
由 sdong 提交于
Summary: CompactRange for universal compaction with num_levels > 1 seems to have a bug. The unit test also has a bug so it doesn't capture the problem. Fix it. Revert the compact range to the logic equivalent to num_levels=1. Always compact all files together. It should also fix DBTest.IncreaseUniversalCompactionNumLevels. The issue was that options.write_buffer_size = 100 << 10 and options.write_buffer_size = 100 << 10 are not used in later test scenarios. So write_buffer_size of 4MB was used. The compaction trigger condition is not anymore obvious as expected. Test Plan: Run the new test and all test suites Reviewers: yhchiang, rven, kradhakrishnan, anthony, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37551
-
由 Igor Canadi 提交于
Summary: Based on feedback from D37083. Are all of these correct? In some spaces it seems like we're doing SetMaxPossibleForUserKey() although we want the smallest possible internal key for user key. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37341
-
由 Igor Canadi 提交于
Summary: Currently open source rocksdb only builds with tcmalloc. This diff first checks if jemalloc is available. If it is, it compiles with jemalloc. If it isn't, it checks for tcmalloc. Test Plan: Tried this out on my Ubuntu virtual machine and confirms that jemalloc is correctly detected and compiled. Reviewers: MarkCallaghan, yhchiang, rven, sdong Reviewed By: sdong Subscribers: adamretter, meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36789
-
由 Igor Canadi 提交于
Summary: Reading CompactionPicker I noticed this dangerous substraction of two unsigned integers. We should assert to mark this as safe. Test Plan: make check Reviewers: anthony, rven, yhchiang, sdong Reviewed By: sdong Subscribers: kradhakrishnan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37041
-
由 Giuseppe Ottaviano 提交于
Summary: This diff implements a new `DB` method `PromoteL0` which moves all files in L0 to a given level skipping compaction, provided that the files have disjoint ranges and all levels up to the target level are empty. This method provides finer-grain control for trivial compactions, and it is useful for bulk-loading pre-sorted keys. Compared to D34797, it does not change the semantics of an existing operation, which can impact existing code. PromoteL0 is designed to work well in combination with the proposed `GetSstFileWriter`/`AddFile` interface, enabling to "design" the level structure by populating one level at a time. Such fine-grained control can be very useful for static or mostly-static databases. Test Plan: `make check` Reviewers: IslamAbdelRahman, philipp, MarkCallaghan, yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37107
-
由 sdong 提交于
Summary: Add more logging to help debugging issues. Test Plan: Run test suites Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37401
-
由 sdong 提交于
Summary: To further distinguish the corruption cases were caused by storage media or in memory states when writing it, add a paranoid check after writing the file to iterate all the rows. Test Plan: Add a new unit test for it Reviewers: rven, igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37335
-
由 Mark Callaghan 提交于
Summary: This is done to avoid having each thread use the same seed between runs of db_bench. Without this we can inflate the OS filesystem cache hit rate on reads for read heavy tests and generally see the same key sequences get generated between teste runs. Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37563
-
- 23 4月, 2015 5 次提交
-
-
由 Venkatesh Radhakrishnan 提交于
Summary: A couple of times on Travis, we have had the thread status say that there were no compactions done and since we assert for it, the test failed. We now fix this by waiting till compaction started. Test Plan: run DBTEST::*PreShutdown* d=/tmp/j; rm -rf $d; seq 200 | parallel --gnu --eta 'd=/tmp/j/d-{}; mkdir -p $d; TEST_TMPDIR=$d ./db_test --gtest_filter=DBTest.PreShutdown* >& '$d'/log-{}' Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37545
-
由 Jim Meyering 提交于
Summary: Without this change, someone on the machine on which I run "make check" could cause me to overwrite arbitrary files owned by me, via a symlink attack. Instead of using a predictable temporary directory and accepting to use a preexisting one, always create a new one using mkdtemp. If $TEST_IOCTL_FRIENDLY_TMPDIR is set and usable, attempt first to find a usable temporary directory therein. If not, or if unusable, then try /var/tmp and /tmp. If none of those is usable abort with a diagnostic. To do that, I added a new class. Its constructor finds a suitable directory or aborts, the sole member prints that directory's name, and the destructor unlinks what should be an empty directory. Note that while the code before this did not remove its temporary directory, there was only one per $UID. Now, there would be at least one per run or one per test, depending on implementation, so it is important to remove them. Test Plan: Run this on a fedora rawhide system, where /tmp is a tmpfs file system, and /var/tmp is ext4. # This gives a diagnostic that /dev/shm is not suitable # and ends up using /var/tmp. TEST_IOCTL_FRIENDLY_TMPDIR=/dev/shm ./env_test # Uses /var/tmp; same as when envvar not set. TEST_IOCTL_FRIENDLY_TMPDIR=/var/tmp ./env_test # Uses /tmp unless it's tmpfs, in which case it gives # a diagnostic and uses /var/tmp. TEST_IOCTL_FRIENDLY_TMPDIR=/tmp ./env_test Reviewers: ljin, rven, igor.sugak, yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37287
-
由 Mark Callaghan 提交于
Summary: This lets the production toolchain libraries get used on devservers and in production. Task ID: #6849362 Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37533
-
由 Mark Callaghan 提交于
Summary: This adds: 1) use of --level_compaction_dynamic_level_bytes=true 2) use of --bytes_per_sync=2M The second is a big win for disks. The first helps in general. This also adds a new test, fillseq with 32kb values to increase the peak ingest and make it more likely that storage limits throughput. Sample outpout from the first 3 tests - https://gist.github.com/mdcallag/e793bd3038e367b05d6f Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37509
-
由 Igor Canadi 提交于
Summary: `echo` correctly interpretes \n on mac, but not on linux. On linux you have to give it `-e` to interpret \n. Unfortunately, `-e` options is not available on Mac. Go back to old way of checking gflags Test Plan: build_tools/build_detect_platform on mac and linux Reviewers: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37515
-
- 22 4月, 2015 2 次提交
-
-
由 Siying Dong 提交于
Add "Tango Me" section in USERS.md
-
由 Min Zhou 提交于
-
- 21 4月, 2015 5 次提交
-
-
由 Igor Canadi 提交于
Summary: Based on comment from D37455 Test Plan: make install after make static_lib Reviewers: meyering Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37461
-
由 Igor Canadi 提交于
Summary: make install fails when there is no shared lib. We need to revert the conditions, which will have the same effect, but without the failure Test Plan: make install after only compiling static library Reviewers: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37455
-
由 Igor Canadi 提交于
Summary: We should send error output to /dev/null Test Plan: none Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37449
-
由 Igor Canadi 提交于
Summary: The current version tries to install librocksdb.so even though it doesn't exist. This version will install librocksdb.so.3.10.0 and then create soft links in place Test Plan: `make static_lib; sudo make install` does not try to install librocksdb.so `make shared_lib; sudo make install` installs one library and 3 symlinks. Before, four libraries were installed Reviewers: sdong, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37425
-
由 Igor Canadi 提交于
Summary: If the system has gflags with both `google` and `gflags` namespaces installed, we try to define GFLAGS as two things. This breaks the compile. Fix: Use `else if` -- try compiling with `google` namespace only if compile with `gflags` failed Test Plan: build_tools/build_detect_platform correctly identifies gflags Reviewers: lgalanis Reviewed By: lgalanis Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37389
-
- 18 4月, 2015 4 次提交
-
-
由 Jim Meyering 提交于
Summary: [NB: this is a prerequisite for the /tmp-abuse-fixing patch] This avoids spurious test failure on Linux systems like Fedora for which /tmp is a tmpfs file system. On a devtmpfs file system, ioctl(fd, FS_IOC_GETVERSION, &version) returns -1 with errno == ENOTTTY, indicating that that ioctl is not supported on such a file system. Do not let this cause test failures, e.g., where env_test would assert that file->GetUniqueId(...) > 0. Before this change, ./env_test would fail these three tests on a fedora rawhide system: [ FAILED ] 3 tests, listed below: [ FAILED ] EnvPosixTest.RandomAccessUniqueID [ FAILED ] EnvPosixTest.RandomAccessUniqueIDConcurrent [ FAILED ] EnvPosixTest.RandomAccessUniqueIDDeletes 3 FAILED TESTS The fix: When support for that ioctl is lacking, skip each affected test. Could be improved by noting which sub-tests are being skipped. Test Plan: run these on F21 and note that they now pass. TEST_TMPDIR=/dev/shm/rdb ./env_test ./env_test Reviewers: ljin, rven, igor.sugak, yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37323
-
由 Igor Canadi 提交于
Summary: Some Mongo+Rocks datasets in Parse's environment are not doing compactions very frequently. During the quiet period (with no IO), we'd like to schedule compactions so that our reads become faster. Also, aggressively compacting during quiet periods helps when write bursts happen. In addition, we also want to compact files that are containing deleted key ranges (like old oplog keys). All of this is currently not possible with CompactRange() because it's single-threaded and blocks all other compactions from happening. Running CompactRange() risks an issue of blocking writes because we generate too much Level 0 files before the compaction is over. Stopping writes is very dangerous because they hold transaction locks. We tried running manual compaction once on Mongo+Rocks and everything fell apart. MarkForCompaction() solves all of those problems. This is very light-weight manual compaction. It is lower priority than automatic compactions, which means it shouldn't interfere with background process keeping the LSM tree clean. However, if no automatic compactions need to be run (or we have extra background threads available), we will start compacting files that are marked for compaction. Test Plan: added a new unit test Reviewers: yhchiang, rven, MarkCallaghan, sdong Reviewed By: sdong Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37083
-
由 Jim Meyering 提交于
Summary: The usage I'm fixing here caused trouble on Fedora 21 when compiling with the current gcc version 4.9.2 20150212 (Red Hat 4.9.2-6) (GCC): db/write_controller_test.cc: In member function ‘virtual void rocksdb::WriteControllerTest_SanityTest_Test::TestBody()’: db/write_controller_test.cc:23:165: error: converting ‘false’ to pointer type for argument 1 of ‘char testing::internal::IsNullLiteralHelper(testing::internal::Secret*)’ [-Werror=conversion-null] ASSERT_EQ(false, controller.IsStopped()); ^ This change was induced mechanically via: git grep -l -E 'ASSERT_EQ\(false'|xargs perl -pi -e 's/ASSERT_EQ\(false, /ASSERT_FALSE(/' git grep -l -E 'ASSERT_EQ\(true'|xargs perl -pi -e 's/ASSERT_EQ\(true, /ASSERT_TRUE(/' Except for the three in utilities/backupable/backupable_db_test.cc for which I ended up reformatting (joining lines) in the result. As for why this problem is exhibited with that version of gcc, and none of the others I've used (from 4.8.1 through gcc-5.0.0 and newer), I suspect it's a bug in F21's gcc that has been fixed in gcc-5.0.0. Test Plan: "make" now succeed on Fedora 21 Reviewers: ljin, rven, igor.sugak, yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37329
-
由 Igor Canadi 提交于
Summary: this is not used anywhere Test Plan: compiles Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37053
-
- 17 4月, 2015 2 次提交
-
-
由 Igor Canadi 提交于
Summary: For some reason reduce_levels is opening the databse with 65.000 levels. This makes ComputeCompactionScore() function terribly slow and the tests is also very slow (20seconds). Test Plan: mr reduce_levels_test now takes 20ms Reviewers: sdong, rven, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37059
-
由 Igor Canadi 提交于
Summary: If ExpandWhileOverlapping() we don't clear inputs. That's a bug introduced by my recent patch https://reviews.facebook.net/D36687. However, we have no tests covering ExpandWhileOverlapping(). I created a task t6771252 to add ExpandWhileOverlapping() tests. Test Plan: make check Reviewers: sdong, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37077
-
- 16 4月, 2015 5 次提交
-
-
由 Igor Canadi 提交于
Added falloc.h in build_detect_platform
-
由 Igor Canadi 提交于
[RocksJava] Fix RateLimiter Tests in 3.10
-
由 fyrz 提交于
-
由 Igor Canadi 提交于
Update USERS.md
-
由 Vlad Balan 提交于
-
- 15 4月, 2015 4 次提交
-
-
由 sdong 提交于
Summary: D36669 introduces a bug that trivial moved data is not going to specific level but the next level, which will incorrectly be level 1 for level 0 compaciton if base level is not level 1. Fixing it by appreciating the output level Test Plan: Run all tests Reviewers: MarkCallaghan, rven, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37119
-
由 sdong 提交于
Summary: Recent change of DBTest.DynamicLevelCompressionPerLevel2 has a bug that the second sync point is not enabled. Fix it. Also add an assert for that. Also, flush compression is not tracked in the test. Add it. Test Plan: Build everything Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37101
-
由 sdong 提交于
Summary: When commiting the sync point interface change, didn't resolve the new occurance of the old interface in rebase. Fix it. Test Plan: Build and see it pass Reviewers: igor, yhchiang, rven, anthony, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37095
-
由 sdong 提交于
SyncPoint to allow a callback with an argument and use it to get DBTest.DynamicLevelCompressionPerLevel2 more straight-forward Summary: Allow users to give a callback function with parameter using sync point, so more complicated verification can be done in tests. Use it in DBTest.DynamicLevelCompressionPerLevel2 so that failures will be more easy to debug. Test Plan: Run all tests. Run DBTest.DynamicLevelCompressionPerLevel2 with valgrind check. Reviewers: rven, yhchiang, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D36999
-
- 14 4月, 2015 4 次提交
-
-
由 Igor Canadi 提交于
Summary: https://reviews.facebook.net/D36963 made the debug build much faster and that triggered failures of CompactFilesOnLevelCompaction test. 3 out of 4 last tests on Jenkins failed. I'm disabling this test temporarily, since we likely know the reason why it's failing and there's already work in progress to address it -- https://reviews.facebook.net/D36225 Test Plan: none Reviewers: sdong, rven, yhchiang, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36993
-
由 Pooya Shareghi 提交于
On Centos 6, you need to explicitely include linux/falloc.h which is whele the FALLOC_FL_* flags are defined. Otherwise, the fallocate() support test defined in build_detect_platform will fail. Signed-off-by: NPooya Shareghi <shareghi@gmail.com>
-
由 Igor Canadi 提交于
Summary: based on discussion on https://reviews.facebook.net/D36969 Test Plan: will let jenkins do its job Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36975
-
由 Mark Callaghan 提交于
Summary: This changes loads to use vector memtable and disable the WAL. This also increases the chance we will see IO bottlenecks during loads which is good to stress test HW. But I also think it is a good way to load data quickly as this is a bulk operation and the WAL isn't needed. The two numbers below are the MB/sec rates for fillseq, bulkload using a skiplist or vector memtable and the WAL enabled or disabled. There is a big benefit from using the vector memtable and WAL disabled. Alas there is also a perf bug in the use of std::sort for ordered input when the vector is flushed. Task is open for that. 112, 66 - skiplist with wal 250, 116 - skiplist without wal 110, 108 - vector with wal 232, 370 - vector without wal Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D36957
-