- 01 3月, 2016 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Fixed two related race conditions in backup creation. (1) CreateNewBackup() uses DB::DisableFileDeletions() to prevent table files from being deleted while it is copying; however, the MANIFEST file could still rotate during this time. The fix is to stop deleting the old manifest in the rotation logic. It will be deleted safely later when PurgeObsoleteFiles() runs (can only happen when file deletions are enabled). (2) CreateNewBackup() did not account for the CURRENT file being mutable. This is significant because the files returned by GetLiveFiles() contain a particular manifest filename, but the manifest to which CURRENT refers can change at any time. This causes problems when CURRENT changes between the call to GetLiveFiles() and when it's copied to the backup directory. To workaround this, I manually forge a CURRENT file referring to the manifest filename returned in GetLiveFiles(). (2) also applies to the checkpointing code, so let me know if this approach is good and I'll make the same change there. Test Plan: new test for roll manifest during backup creation. running the test before this change: $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation ... IO error: /tmp/rocksdbtest-9383/backupable_db/MANIFEST-000001: No such file or directory running the test after this change: $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation ... [ RUN ] BackupableDBTest.ChangeManifestDuringBackupCreation [ OK ] BackupableDBTest.ChangeManifestDuringBackupCreation (2836 ms) Reviewers: IslamAbdelRahman, anthony, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54711
-
由 Igor Canadi 提交于
build: Improve -momit-leaf-frame-pointer usage
-
- 29 2月, 2016 1 次提交
-
-
由 James Page 提交于
-momit-leaf-frame-pointer is only supported on certain archs. Detect this automatically based on which flags the compiler understands.
-
- 27 2月, 2016 1 次提交
-
-
由 sdong 提交于
Summary: Based on thread scheduling, DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels can fail to flush enough files to trigger expected compactions. Fix it by waiting for flush after inserting each key. There are failrue reported: db/db_universal_compaction_test.cc:1134: Failure Expected: (NumTableFilesAtLevel(options.num_levels - 1, 1)) > (0), actual: 0 vs 0 but I can't repro it. Try to fix the bug and see whether it goes away. Test Plan: Run the test multiple time. Reviewers: IslamAbdelRahman, anthony, andrewkr, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54747
-
- 26 2月, 2016 1 次提交
-
-
由 agiardullo 提交于
Summary: This was fixed by 0c2bd5cb Test Plan: n/a Reviewers: gabijs Reviewed By: gabijs Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54753
-
- 25 2月, 2016 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: As titled. This fixes the tsan error caused by logger_ being used in backup_engine_'s destructor. It does not fix the transient unit test failure, which is caused by MANIFEST file changing while backup is happening. Test Plan: verified the tsan error no longer happens on either success or failure. $ COMPILE_WITH_TSAN=1 make -j32 backupable_db_test $ while ./backupable_db_test --gtest_filter=BackupableDBTest.CorruptionsTest ; do : ; done Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54669
-
由 sdong 提交于
Summary: Add a test case in ColumnFamilyTest.ReadDroppedColumnFamily to make sure existing iterator is not impacted by column family dropping. Test Plan: N/A Reviewers: igor, yhchiang, anthony, andrewkr, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54657
-
- 24 2月, 2016 2 次提交
-
-
由 sdong 提交于
Summary: DBImpl::SyncWAL() releases db mutex before calling DBImpl::MarkLogsSynced(), while inside DBImpl::MarkLogsSynced() we assert there is none or one outstanding log file. However, a memtable switch can happen in between and causing two or outstanding logs there, failing the assert. The diff adds a unit test that repros the issue and fix the assert so that the unit test passes. Test Plan: Run the new tests. Reviewers: anthony, kolmike, yhchiang, IslamAbdelRahman, kradhakrishnan, andrewkr Reviewed By: andrewkr Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54621
-
由 Yi Wu 提交于
Summary: similar to D52809 add option to exclude zero counters. Test Plan: [yiwu@dev4504.prn1 ~/rocksdb] ./iostats_context_test [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from IOStatsContextTest [ RUN ] IOStatsContextTest.ToString [ OK ] IOStatsContextTest.ToString (0 ms) [----------] 1 test from IOStatsContextTest (0 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (0 ms total) [ PASSED ] 1 test. Reviewers: anthony, yhchiang, andrewkr, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54591
-
- 23 2月, 2016 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: There was a race condition in the test where the rolling thread acquired the mutex before the flush thread pinned the logger. Rather than add more complicated synchronization to fix it, I followed Siying's suggestion to use SyncPoint in the test code. Comments in the LoadDependency() invocation explain the reason for each of the sync points. Test Plan: Ran test 1000 times for tsan/asan. Will wait for all sandcastle tests to finish before committing since this is a tricky test. Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54615
-
由 Yueh-Hsuan Chiang 提交于
Summary: This reverts commit 73c31377, which mistakenly reverts 73c31377 that fixes a bug when both whole_key_filtering and prefix_extractor are set Test Plan: revert the patch Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52707
-
由 Mike Kolupaev 提交于
Summary: We started getting two kinds of crashes since we started using `DB::CompactFiles()`: (1) `CompactFiles()` fails saying something like "/data/logdevice/4440/shard12/012302.sst: No such file or directory", and presumably makes DB read-only, (2) DB fails to open saying "Corruption: Can't access /267000.sst: IO error: /data/logdevice/4440/shard1/267000.sst: No such file or directory". AFAICT, both can be explained by background thread deleting compaction output as "obsolete" while it's being written, before it's committed to manifest. If it ends up committed to the manifest, we get (2); if compaction notices the disappearance and fails, we get (1). The internal tasks t10068021 and t10134177 have some details about the investigation that led to this. Test Plan: `make -j check`; the new test fails to reopen the DB without the fix Reviewers: yhchiang Reviewed By: yhchiang Subscribers: dhruba, sdong Differential Revision: https://reviews.facebook.net/D54561
-
- 20 2月, 2016 6 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Relax the check condition of prefix_extractor in CheckOptionsCompatibility by allowing changing value from non-nullptr to nullptr or nullptr to non-nullptr. Test Plan: options_test options_util_test Reviewers: sdong, anthony, IslamAbdelRahman, kradhakrishnan, gunnarku Reviewed By: gunnarku Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54477
-
由 Igor Canadi 提交于
Implement ConsistentChildrenAttribute on Windows
-
由 Dmitri Smirnov 提交于
by using default implementation for now as it works.
-
由 sdong 提交于
Summary: Recent change break thread_local_test by introducing exception, which is disabled in LITE build. Fix it by disabling exception handling in LITE build. Test Plan: Build with both of LITE and non-LITE Reviewers: anthony, IslamAbdelRahman, yhchiang, kradhakrishnan, andrewkr Reviewed By: andrewkr Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54513
-
由 Igor Canadi 提交于
This addresses build issues on Windows
-
-
- 19 2月, 2016 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Previously I just slept until the flush_thread was "probably" ready since proper synchronization in test cases seemed like overkill. But then tsan complained about it, so I did the synchronization (mostly) properly now. Test Plan: $ COMPILE_WITH_TSAN=1 make -j32 auto_roll_logger_test $ ./auto_roll_logger_test Reviewers: anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54399
-
由 Islam AbdelRahman 提交于
Summary: I have introduced max_allowed_space_ but did not initialize it Test Plan: make check Reviewers: sdong, yhchiang, anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D54357
-
- 18 2月, 2016 6 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Remove the SyncPoint usage in the destructor of PosixEnv as none of any active tests is using it. SyncPoint is a test-only utility class, and it's a static varible. As a result, using SyncPoint in the destructor of PosixEnv will make default Env depends on SyncPoint. Removing such dependency could solve the problem crash issue only reproducable in Mac environment. Test Plan: OPT=-DTRAVIS V=1 make -j4 check on Mac environment Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54333
-
由 Islam AbdelRahman 提交于
Summary: Introude SstFileManager::SetMaxAllowedSpaceUsage() that can be used to limit the maximum space usage allowed for RocksDB. When this limit is exceeded WriteImpl() will fail and return Status::Aborted() Test Plan: unit testing Reviewers: yhchiang, anthony, andrewkr, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D53763
-
由 Andrew Kryczka 提交于
Summary: For GetLogFileSize() and Flush(), they previously did not follow the synchronization pattern for accessing logger_. This meant ResetLogger() could cause logger_ destruction while the unsynchronized functions were accessing it, causing a segfault. Also made the mutex instance variable mutable so we can preserve GetLogFileSize()'s const-ness. Test Plan: new test case, it's quite ugly because both threads need to access one of the functions with SyncPoints (PosixLogger::Flush()), and also special handling is needed to prevent the mutex and sync points from conflicting. Reviewers: kradhakrishnan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54237
-
由 Andrew Kryczka 提交于
Summary: I broke it in D53781. Test Plan: tried the same code in util/env_posix.cc and it compiled successfully Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54303
-
由 Jonathan Wiepert 提交于
Test Plan: pushed remote as D53991_appveyor_test branch. Appveyor run: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.651 shows the file is now found and the run is back to the state before the initial change: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.620 Reviewers: sdong, igor Reviewed By: igor Subscribers: igor, dhruba Differential Revision: https://reviews.facebook.net/D54309
-
由 Igor Canadi 提交于
fix ios build error
-
- 17 2月, 2016 9 次提交
-
-
由 fengjian 提交于
-
由 agiardullo 提交于
Summary: Broke transaction locking in 4.4 in D52197. Will cherry-pick this change into 4.4 (which hasn't yet been fully released). Repro'd using db_bench. Test Plan: unit tests and db_Bench Reviewers: sdong, yhchiang, kradhakrishnan, ngbronson Reviewed By: ngbronson Subscribers: ngbronson, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54021
-
由 Yueh-Hsuan Chiang 提交于
Summary: Improve the documentation of LoadLatestOptions Test Plan: No code change Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54087
-
由 reid horuff 提交于
Summary: Currently write_with_callback_test does not test with WAL syncing enabled. This addresses that. Test Plan: write_with_callback_test Reviewers: anthony Reviewed By: anthony Subscribers: leveldb, dhruba, hermanlee4 Differential Revision: https://reviews.facebook.net/D54255
-
由 reid horuff 提交于
Summary: There is an issue in DBImpl::WriteImpl where if an empty writebatch comes in and sync=true then the logs will be marked as being synced yet the sync never actually happens because there is no data in the writebatch. This causes the next incoming batch to hang while waiting for the logs to complete syncing. This fix syncs logs even if the writebatch is empty. Test Plan: DoubleEmptyBatch unit test in transaction_test. Reviewers: yoshinorim, hermanlee4, sdong, ngbronson, anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D54057
-
由 Jonathan Wiepert 提交于
Test Plan: Built and ran with gflags: % ./db_bench LevelDB: version 4.5 Date: Tue Feb 16 12:04:23 2016 CPU: 40 * Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz ... And without gflags: % ./db_bench Please install gflags to run rocksdb tools % Reviewers: sdong, igor Reviewed By: igor Subscribers: igor, dhruba Differential Revision: https://reviews.facebook.net/D54243
-
由 Igor Canadi 提交于
Test Plan: I couldn't repro, but I hope this fixes it. See the error here: https://evergreen.mongodb.com/task_log_raw/mongodb_mongo_master_ubuntu1404_rocksdb_compile_6e9fd902d5cb25aef992363efa128640affd5196_16_02_11_04_33_37/0?type=T Reviewers: yhchiang, andrewkr, sdong, anthony Reviewed By: anthony Subscribers: meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54123
-
由 Mike Kolupaev 提交于
Summary: We've hit it today. Test Plan: `make -j check`; didn't reproduce the issue Reviewers: yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D54219
-
由 Igor Canadi 提交于
fixes typo in options logging
-
- 16 2月, 2016 2 次提交
-
-
由 Jonathan Wiepert 提交于
Summary: Isolate db_bench functionality from main so custom benchmark code can be written and managed Test Plan: Tested commands ./build_tools/regression_build_test.sh ./db_bench --db=/tmp/rocksdbtest-12321/dbbench --stats_interval_seconds=1 --num=1000 ./db_bench --db=/tmp/rocksdbtest-12321/dbbench --stats_interval_seconds=1 --num=1000 --reads=500 --writes=500 ./db_bench --db=/tmp/rocksdbtest-12321/dbbench --stats_interval_seconds=1 --num=1000 --merge_keys=100 --numdistinct=100 --num_column_families=3 --num_hot_column_families=1 ./db_bench --stats_interval_seconds=1 --num=1000 --bloom_locality=1 --seed=5 --threads=5 ./db_bench --duration=60 --value_size=50 --seek_nexts=10 --reverse_iterator=true --usee_uint64_comparator=true --batch-size=5 ./db_bench --duration=60 --value_size=50 --seek_nexts=10 --reverse_iterator=true --use_uint64_comparator=true --batch_size=5 ./db_bench --duration=60 --value_size=50 --seek_nexts=10 --reverse_iterator=true --usee_uint64_comparator=true --batch-size=5 Test Results - https://phabricator.fb.com/P56130387 Additional tests for: ./db_bench --duration=60 --value_size=50 --seek_nexts=10 --reverse_iterator=true --use_uint64_comparator=true --batch_size=5 --key_size=8 --merge_operator=put ./db_bench --stats_interval_seconds=1 --num=1000 --bloom_locality=1 --seed=5 --threads=5 --merge_operator=uint64add Results: https://phabricator.fb.com/P56130607 Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D53991
-
由 John Dennison 提交于
-
- 13 2月, 2016 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Previously compilation failed when ROCKSDB_NO_FBCODE=1 because fcntl.h wasn't included for open(). Related issue: https://github.com/facebook/rocksdb/issues/977 Test Plan: verified below command works now: $ make clean && ROCKSDB_NO_FBCODE=1 ROCKSDB_DISABLE_FALLOCATE=1 make -j32 env_test Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54135
-
由 krad 提交于
Summary: Users are confused on how to get the parallel compilation going. This can help wire the parallelism. Test Plan: Run manually Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D53931
-
由 Andrew Kryczka 提交于
Summary: as titled, this will prevent the error that was printed because test_names was evaluated before db_test was built. Test Plan: verified below command works and no longer prints errors: $ make release -j32 verified below command still finds the right tests: $ make J=32 parallel_check Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54117
-