- 16 9月, 2015 5 次提交
-
-
由 krad 提交于
Summary: Timeout needs to be set for command and job for it to work. Test Plan: Manual Reviewers: sdong igor CC: levledb@ Task ID: #6968635 Blame Rev:
-
由 Andres Noetzli 提交于
Summary: The default behavior for atomic operations is sequentially consistent ordering which is not needed for simple counters (see: http://en.cppreference.com/w/cpp/atomic/memory_order). Change the memory order to std::memory_order_relaxed for better performance. Test Plan: make clean all check Reviewers: rven, anthony, yhchiang, aekmekji, sdong, MarkCallaghan, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46953
-
由 Yoshinori Matsunobu 提交于
Summary: There are some use cases in MyRocks to compare two slices and to return the first byte where they differ. It may be useful to add it as a RocksDB Slice function. Test Plan: db_test Reviewers: sdong, rven, igor Reviewed By: igor Subscribers: jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D46935
-
由 Igor Canadi 提交于
Fix `integer overflow in expression' error
-
由 Igor Canadi 提交于
Fix printf format for size_t
-
- 15 9月, 2015 12 次提交
-
-
由 Dmitry Marakasov 提交于
-
由 Dmitry Marakasov 提交于
-
由 sdong 提交于
DBImpl::FindObsoleteFiles() shouldn't release mutex between getting min_pending_output and scanning files Summary: Releasing mutex between getting min_pending_output and scanning files may cause min_pending_output to be max but some non-final files are found in file scanning, ending up with deleting wrong files. As a recent regression, mutex can be released while waiting for log sync. We move it to after file scanning. Test Plan: Run all existing tests. Don't think it is easy to write a unit test. Maybe we should find a way to assert lock not released so that we can have some test verification for similar cases. Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, kolmike, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D46899
-
由 Islam AbdelRahman 提交于
Summary: Valgrind is complaining because we are using hard_rate_limit (when serializing the options) without being initialized http://our.intern.facebook.com/intern/sandcastle/3962140295/77533971/ Test Plan: run the test under valgrind Reviewers: kradhakrishnan, yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46929
-
由 Islam AbdelRahman 提交于
Summary: Skip these tests under ROCKSDB_LITE compaction_job_stats_test corruption_test transactions/transaction_test Test Plan: compile using ROCKSDB_LITE Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46923
-
由 Andres Noetzli 提交于
Summary: Travis is failing due to missing gflags (https://travis-ci.org/facebook/rocksdb/jobs/80307921). Adding libgflags-dev should help. Test Plan: Run tests on Travis. Reviewers: rven, anthony, yhchiang, aekmekji, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46905
-
由 sdong 提交于
Summary: Add an option to stop writes if compaction lefts behind. If estimated pending compaction bytes is more than threshold specified by options.hard_pending_compaction_bytes_liimt, writes will stop until compactions are cleared to under the threshold. Test Plan: Add unit test DBTest.HardLimit Reviewers: rven, kradhakrishnan, anthony, IslamAbdelRahman, yhchiang, igor Reviewed By: igor Subscribers: MarkCallaghan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D45999
-
由 Islam AbdelRahman 提交于
Summary: Fix compaction_job_stats under ROCKSDB_LITE Test Plan: compile using ROCKSDB_LITE Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46887
-
由 Siying Dong 提交于
Refactor to support file_reader_writer on Windows.
-
由 krad 提交于
Summary: Command level timeouts are not working. Moving it to step level. Test Plan: Manual testing sending commands Reviewers: sdong igor CC: leveldb@ Task ID: #6968635 Blame Rev:
-
由 Andres Noetzli 提交于
Summary: Commit c67d2068 did not fix all test conditions which use Arena::MemoryAllocatedBytes() (see Travis failure https://travis-ci.org/facebook/rocksdb/jobs/79957700). The assumption of that commit was that aligned allocations do not call Arena::AllocateNewBlock(), so malloc_usable_block_size() would not be used for Arena::MemoryAllocatedBytes(). However, there is a code path where Arena::AllocateAligned() calls AllocateFallback() which in turn calls Arena::AllocateNewBlock(), so Arena::MemoryAllocatedBytes() may return a greater value than expected even for aligned requests. Test Plan: make arena_test && ./arena_test Reviewers: rven, anthony, yhchiang, aekmekji, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46869
-
由 Ari Ekmekji 提交于
Summary: Although there are currently counters to keep track of the stall caused by having too many L0 files, there is no distinction as to whether when that stall occurs either (A) L0-L1 compaction is taking place to try and mitigate it, or (B) no L0-L1 compaction has been scheduled at the moment. This diff adds a counter for (A) so that the nature of L0 stalls can be better understood. Test Plan: make all && make check Reviewers: sdong, igor, anthony, noetzli, yhchiang Reviewed By: yhchiang Subscribers: MarkCallaghan, dhruba Differential Revision: https://reviews.facebook.net/D46749
-
- 12 9月, 2015 10 次提交
-
-
由 agiardullo 提交于
Summary: Transaction::RollbackToSavePoint() will now release any locks that were taken since the previous SavePoint. To do this cleanly, I moved tracked_keys_ management into TransactionBase. Test Plan: New Transaction test. Reviewers: igor, rven, sdong Reviewed By: sdong Subscribers: dhruba, spetrunia, leveldb Differential Revision: https://reviews.facebook.net/D46761
-
由 Dmitri Smirnov 提交于
Fix compilation issues on GCC/CLANG Address Windows Release test build issues due to Sync
-
由 krad 提交于
Summary: Fixed - Added timeouts for crash tests. They take around 9hrs. - Added oncall so we can get notifications for timeout etc. - Fixed a bug in the valgrind script - Cosmetic fix Test Plan: Manual run Reviewers: sdong igor CC: leveldb@ Task ID: #6968635 Blame Rev:
-
由 krad 提交于
Summary: Fix the issue where compilation error will not result in log file not found error Test Plan: None Reviewers: sdong igor CC: leveldb@ Task ID: #6968635 Blame Rev:
-
由 Andres Noetzli 提交于
Summary: ArenaTest.MemoryAllocatedBytes on Travis failed: https://travis-ci.org/facebook/rocksdb/jobs/79887849 . This is probably due to malloc_usable_size() returning a value greater than the requested size. From the man page: The value returned by malloc_usable_size() may be greater than the requested size of the allocation because of alignment and minimum size constraints. Although the excess bytes can be overwritten by the application without ill effects, this is not good programming practice: the number of excess bytes in an allocation depends on the underlying implementation. Test Plan: make arena_test && ./arena_test Reviewers: rven, anthony, yhchiang, aekmekji, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46743
-
由 Andres Noetzli 提交于
Summary: RocksDB debug version failed to build under gcc-4.8.1 on sandcastle with the following error: ``` db/db_compaction_filter_test.cc:570:33: error: ‘snapshot’ may be used uninitialized in this function [-Werror=maybe-uninitialized] ``` Test Plan: make db_compaction_filter_test && ./db_compaction_filter_test Reviewers: rven, anthony, yhchiang, aekmekji, igor, sdong Reviewed By: igor, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46725
-
由 Manuel Ung 提交于
Summary: There are currently no statistics on seeks, only on gets. This adds the following counters: rocksdb.number.db.seek rocksdb.number.db.next rocksdb.number.db.prev (number of calls) rocksdb.db.iterate.bytes.read (number of bytes read from key + value using seek/next/prev) rocksdb.number.keys.seek.found rocksdb.number.keys.next.found rocksdb.number.keys.prev.found (number of calls where seek/next/prev found a value) Test Plan: ./db_bench -statistics -benchmarks fillrandom,seekrandom -seek_nexts 5 ./db_bench -statistics -benchmarks fillrandom,seekrandom -seek_nexts 5 -reverse_iterator Reviewers: yhchiang, rven, kradhakrishnan, IslamAbdelRahman, MarkCallaghan, sdong, igor Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46605
-
由 Islam AbdelRahman 提交于
Summary: Refactoring NewTableReader to accept TableReaderOptions This will make it easier to add new options in the future, for example in this diff https://reviews.facebook.net/D46071 Test Plan: run existing tests Reviewers: igor, yhchiang, anthony, rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46179
-
由 Andres Noetzli 提交于
Summary: During the refactoring, the condition that makes sure that compaction filters are only applied to records newer than the latest snapshot got butchered. This patch fixes the condition and adds a test case. Test Plan: make db_compaction_filter_test && ./db_compaction_filter_test Reviewers: rven, anthony, yhchiang, sdong, aekmekji, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46707
-
由 Dmitri Smirnov 提交于
Summary. A change https://reviews.facebook.net/differential/diff/224721/ Has attempted to move common functionality out of platform dependent code to a new facility called file_reader_writer. This includes: - perf counters - Buffering - RateLimiting However, the change did not attempt to refactor Windows code. To mitigate, we introduce new quering interfaces such as UseOSBuffer(), GetRequiredBufferAlignment() and ReaderWriterForward() for pure forwarding where required. Introduce WritableFile got a new method Truncate(). This is to communicate to the file as to how much data it has on close. - When space is pre-allocated on Linux it is filled with zeros implicitly, no such thing exist on Windows so we must truncate file on close. - When operating in unbuffered mode the last page is filled with zeros but we still want to truncate. Previously, Close() would take care of it but now buffer management is shifted to the wrappers and the file has no idea about the file true size. This means that Close() on the wrapper level must always include Truncate() as well as wrapper __dtor should call Close() and against double Close(). Move buffered/unbuffered write logic to the wrapper. Utilize Aligned buffer class. Adjust tests and implement Truncate() where necessary. Come up with reasonable defaults for new virtual interfaces. Forward calls for RandomAccessReadAhead class to avoid double buffering and locking (double locking in unbuffered mode on WIndows).
-
- 11 9月, 2015 10 次提交
-
-
由 Igor Canadi 提交于
Summary: With old travis infrastructure, we couldn't run the whole test suite without crashing. Now we transfered to the new architecture and are able to run full `make check` instead of just db_test. Test Plan: https://travis-ci.org/facebook/rocksdb/builds/79591564 This test has failed, true, but it's actual problem with tests: * t8316104 -- Failed ColumnFamilyTest.ReadDroppedColumnFamily and also https://github.com/facebook/rocksdb/issues/673 * Too many open files: db/db_universal_compaction_test.cc:514: Failure Put(1, Key(i % num_keys), Key(i)) IO error: /tmp/rocksdbtest-501/db_universal_compaction_prallel_test/000331.sst: Too many open files Reviewers: sdong, anthony, kradhakrishnan, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46545
-
由 Andres Noetzli 提交于
Summary: The current build is failing on some platforms due to an __unused__ attribute. This patch prevents the problem by using a pattern similar to MergeHelper (assert not on the variable but inside a condition that uses the variable). We should have better error handling in both cases in the future. Test Plan: make clean all check Reviewers: rven, anthony, yhchiang, sdong, igor, aekmekji Reviewed By: aekmekji Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46623
-
由 Ari Ekmekji 提交于
Summary: The test SuggestCompactRangeNoTwoLevel0Compactions in DBCompactionTest fails when there are parallel L0-L1 compactions taking place because the test makes sure that only one compaction involving L0 takes place at any given time (since before having parallel compactions this was impossible). I changed the test to only run with DBOptions.max_subcompactions=1 so as to not hit this issue which is not a correctness issue but just an inherent changing of assumptions after introducing parallel compactions. This failed after landing https://reviews.facebook.net/D43269#inline-321303 so now this should fix it Test Plan: make all && make check Reviewers: yhchiang, igor, anthony, noetzli, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D46617
-
由 Andres Noetzli 提交于
Summary: Builder and CompactionJob share a lot of fairly complex code. This patch refactors this code into a separate class, the CompactionIterator. Because the shared code is fairly complex, this patch hopefully improves maintainability. While there are is a lot of potential for further improvements, the patch is intentionally pretty close to the original structure because the change is already complex enough. Test Plan: make clean all check && ./db_stress Reviewers: rven, anthony, yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46197
-
由 krad 提交于
Summary: Added more jobs and refactored code express the jobs more cleanly Test Plan: Manual test Reviewers: igor sdong CC: leveldb@ Task ID: #6968635 Blame Rev:
-
由 Igor Canadi 提交于
Summary: ReadDroppedColumnFamily is consistently failing in Travis CI environment (can't repro locally). I suspect it might be failing with non-OK status. This diff will give us more info about the failure. Test Plan: none Reviewers: sdong, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: kradhakrishnan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46611
-
由 Ari Ekmekji 提交于
Summary: Up to this point, the subcompactions that make up a compaction job have been divided based on the key range of the L1 files, and each subcompaction has handled the key range of only one file. However DBOption.max_subcompactions allows the user to designate how many subcompactions at most to perform. This patch updates the CompactionJob::GetSubcompactionBoundaries() to determine these divisions accordingly based on that option and other input/system factors. The current approach orders the starting and/or ending keys of certain compaction input files and then generates a histogram to approximate the size covered by the key range between each consecutive pair of keys. Then it groups these ranges into groups so that the sizes are approximately equal to one another. The approach has also been adapted to work for universal compaction as well instead of just for level-based compaction as it was before. These subcompactions are then executed in parallel by locally spawning threads, one for each. The results are then aggregated and the compaction completed. Test Plan: make all && make check Reviewers: yhchiang, anthony, igor, noetzli, sdong Reviewed By: sdong Subscribers: MarkCallaghan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43269
-
由 krad 提交于
Summary: The current code, considers data to be consistent if the record checksum passes. We do have customer issues where the record checksum passed but the data was incomprehensible. There is no way to get out of this error case since all WAL recovery model will consider this error as unrelated to WAL. Relaxing the definition and including errors while inserting to memtable as WAL errors and handing them as per the recovery level. Test Plan: Used customer dump to verify the fix for different level. The db opens for kSkipAnyCorruptedRecords and kPointInTimeRecovery, but fails for kAbsoluteConsistency and kTolerateCorruptedTailRecords. Reviewers: sdon igor CC: leveldb@ Task ID: #7918721 Blame Rev:
-
由 sdong 提交于
Summary: DBTest.ReadLatencyHistogramByLevel was not written as expected. After writes, reads aren't guaranteed to hit data written. It was not expected. Fix it. Test Plan: Run the test multiple times Reviewers: IslamAbdelRahman, rven, anthony, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D46587
-
由 Igor Canadi 提交于
Summary: We should never set max_open_files to be bigger than the system's ulimit. Otherwise we will get "Too many open files" errors. See an example in this Travis run: https://travis-ci.org/facebook/rocksdb/jobs/79591566 Test Plan: make check I will also verify that max_max_open_files is reasonable. Reviewers: anthony, kradhakrishnan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46551
-
- 10 9月, 2015 3 次提交
-
-
由 Igor Canadi 提交于
Run travis tests on OS X
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Upgrade travis to new architecture
-