- 24 8月, 2016 6 次提交
-
-
由 Aaron Gao 提交于
Summary: fixed data race described in https://github.com/facebook/rocksdb/issues/1267 and add regression test Test Plan: ./table_test --gtest_filter=BlockBasedTableTest.NewIndexIteratorLeak make all check -j64 core dump before fix. ok after fix. Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: igor, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62361
-
由 Yi Wu 提交于
Summary: We used to allow insert into full block cache as long as `strict_capacity_limit=false`. This diff further restrict insert to full cache if caller don't intent to hold handle to the cache entry after insert. Hope this diff fix the assertion failure with db_stress: https://our.intern.facebook.com/intern/sandcastle/log/?instance_id=211853102&step_id=2475070014 db_stress: util/lru_cache.cc:278: virtual void rocksdb::LRUCacheShard::Release(rocksdb::Cache::Handle*): Assertion `lru_.next == &lru_' failed. The assertion at lru_cache.cc:278 can fail when an entry is inserted into full cache and stay in LRU list. Test Plan: make all check Reviewers: IslamAbdelRahman, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62325
-
由 Yi Wu 提交于
Summary: Add option to block based table to insert index/filter blocks to block cache with priority. Combined with LRUCache with high_pri_pool_ratio, we can reserved space for index/filter blocks, make them less likely to be evicted. Depends on D61977. Test Plan: See unit test. Reviewers: lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D62241
-
由 Yi Wu 提交于
Summary: ..to unblock TSAN contbuild. Test Plan: COMPILE_WITH_TSAN=1 make db_stress -j64 ./db_stress --use_clock_cache=true Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62397
-
由 Andrew Kryczka 提交于
Summary: make the variables static so capturing is unnecessary since I couldn't find a portable way to capture variables in a lambda that's converted to a C-style pointer-to-function. Test Plan: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.1658 Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62403
-
由 Andrew Kryczka 提交于
Summary: there was an error when accessing kItersPerThread in the lambda: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.1654 Test Plan: doitlive Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62379
-
- 23 8月, 2016 4 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This function allows the user to provide a custom function to fold all threads' local data. It will be used in my next diff for aggregating statistics stored in thread-local data. Note the test case uses atomics as thread-local values due to the synchronization requirement (documented in code). Test Plan: unit test Reviewers: yhchiang, sdong, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62049
-
由 Adam Retter 提交于
* Rename RocksDB#remove -> RocksDB#delete to match C++ API; Added deprecated versions of RocksDB#remove for backwards compatibility. * Add missing experimental feature RocksDB#singleDelete
-
由 Adam Retter 提交于
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
-
由 Andrew Kryczka 提交于
Summary: value is not an InternalKey, we do not need to decode it Test Plan: setup: $ ldb put --create_if_missing=true k v $ ldb put --db=./tmp --create_if_missing k v $ ldb compact --db=./tmp before: $ sst_dump --command=raw --file=./tmp/000004.sst ... terminate called after throwing an instance of 'std::length_error' after: $ ./sst_dump --command=raw --file=./tmp/000004.sst $ cat tmp/000004_dump.txt ... ASCII k : v ... Reviewers: sdong, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62301
-
- 20 8月, 2016 4 次提交
-
-
由 Yi Wu 提交于
Summary: Add mid-point insertion functionality to LRU cache. Caller of `Cache::Insert()` can set an additional parameter to make a cache entry have higher priority. The LRU cache will reserve at most `capacity * high_pri_pool_pct` bytes for high-pri cache entries. If `high_pri_pool_pct` is zero, the cache degenerates to normal LRU cache. Context: If we are to put index and filter blocks into RocksDB block cache, index/filter block can be swap out too early. We want to add an option to RocksDB to reserve some capacity in block cache just for index/filter blocks, to mitigate the issue. In later diffs I'll update block based table reader to use the interface to cache index/filter blocks at high priority, and expose the option to `DBOptions` and make it dynamic changeable. Test Plan: unit test. Reviewers: IslamAbdelRahman, sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D61977
-
由 Islam AbdelRahman 提交于
Summary: Update SstFileWriter to use user TablePropertiesCollectors that are passed in Options Test Plan: unittests Reviewers: sdong Reviewed By: sdong Subscribers: jkedgar, andrewkr, hermanlee4, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D62253
-
由 Wanning Jiang 提交于
Summary: 1. Range Deletion Tombstone structure 2. Modify Add() in table_builder to make it usable for adding range del tombstones 3. Expose NewTombstoneIterator() API in table_reader Test Plan: table_test.cc (now BlockBasedTableBuilder::Add() only accepts InternalKey. I make table_test only pass InternalKey to BlockBasedTableBuidler. Also test writing/reading range deletion tombstones in table_test ) Reviewers: sdong, IslamAbdelRahman, lightmark, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61473
-
由 Yi Wu 提交于
Summary: Clock-based cache implemenetation aim to have better concurreny than default LRU cache. See inline comments for implementation details. Test Plan: Update cache_test to run on both LRUCache and ClockCache. Adding some new tests to catch some of the bugs that I fixed while implementing the cache. Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61647
-
- 19 8月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Splitting the makefile part of D55581. Test Plan: make all check -j32 ROCKSDB_FBCODE_BUILD_WITH_481=1 make all check -j32 ROCKSDB_NO_FBCODE=1 make all check -j32 export TBB_BASE=/mnt/gvfs/third-party2/tbb/afa54b33cfcf93f1d90a3160cdb894d6d63d5dca/4.0_update2/gcc-4.9-glibc-2.20/e9936bf; ROCKSDB_NO_FBCODE=1 CFLAGS="-I $TBB_BASE/include" LDFLAGS="-L $TBB_BASE/lib -Wl,-rpath=$TBB_BASE/lib" make all check -j32 Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: kradhakrishnan, yhchiang, IslamAbdelRahman, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56979
-
- 18 8月, 2016 2 次提交
-
-
由 Jay 提交于
close #1262
-
由 Alexander Jipa 提交于
fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219) CMake - Use a platform-neutral mkdir function
-
- 17 8月, 2016 2 次提交
-
-
由 Dmitri Smirnov 提交于
* Create rate limiter using factory function in the test. * Convert function local statics in option helper to a C array that does not perform dynamic memory allocation. This is helpful when you try to memory isolate different DB instances.
-
由 Yi Wu 提交于
Summary: ... so that I can include the header and create LRUCache specific tests for D61977 Test Plan: make check Reviewers: lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62145
-
- 16 8月, 2016 9 次提交
-
-
由 Anirban Rahut 提交于
Summary: Added 2 statistics in compaction job statistics, to identify if single deletes are not meeting a matching key (fallthrough) or single deletes are meeting a merge, delete or another single delete (i.e. not the expected case of put). Test Plan: Tested the statistics using write_stress and compaction_job_stats_test Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61749
-
由 Andrew Kryczka 提交于
Summary: Add API to WriteBatch to store range deletions in its buffer which are later added to memtable. In the WriteBatch buffer, a range deletion is encoded as "<optype><CF ID (optional)><begin key><end key>". With this diff, the range tombstones are stored inline with the data in the memtable. It's useful for now because the test cases rely on the data being accessible via memtable. My next step is to store range tombstones in a separate area in the memtable. Test Plan: unit tests Reviewers: IslamAbdelRahman, sdong, wanning Reviewed By: wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61401
-
由 Andrew Kryczka 提交于
Summary: We were frequently seeing a race between SyncPoint::Process() and SyncPoint::~SyncPoint() (e.g., https://our.intern.facebook.com/intern/sandcastle/log/?instance_id=207289975&step_id=2412725431). The issue was marked_thread_id_ gets deleted when the main thread is exiting and simultaneously background threads may access it. We can prevent this race condition by checking whether sync points are disabled (assuming the test terminates with them disabled) before attempting to access that member. I do not understand why accesses to other members (mutex_ and enabled_) are ok but anyways the test no longer fails tsan. Test Plan: ran tests Reviewers: sdong, yhchiang, IslamAbdelRahman, yiwu, wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62133
-
由 Islam AbdelRahman 提交于
Summary: Fix the test by releasing the last snapshot Test Plan: run the test under valgrind Reviewers: andrewkr, yiwu, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62091
-
由 Islam AbdelRahman 提交于
Summary: Fix the java build Test Plan: make rocksdbjava -j64 Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62097
-
由 Mark Callaghan 提交于
Summary: Changes compaction IO stats to be printed once per interval rather than once per interval per thread. https://github.com/facebook/rocksdb/issues/1276 Task ID: #12698508 Test Plan: run db_bench Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62067
-
由 sdong 提交于
Summary: "Batch" is ambiguous in this context. It can mean "write batch" or commit group. Change it to commit group to be clear. Test Plan: Build Reviewers: MarkCallaghan, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62055
-
由 Edouard A 提交于
Users shouldn't reply on -> users shouldn't rely on
-
由 Yueh-Hsuan Chiang 提交于
Summary: Env holds a pointer of ThreadStatusUpdater, which will be deleted when Env is deleted. However, in case a rocksdb database is deleted after Env is deleted. Then this will introduce a free-after-use of this ThreadStatusUpdater. This patch fix this by never deleting the ThreadStatusUpdater in Env, which is in general safe as Env is a singleton in most cases. Test Plan: thread_list_test Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59187
-
- 13 8月, 2016 2 次提交
-
-
由 Philipp Unterbrunner 提交于
Summary: Added min/max/avg data block size output to sst_dump. Output was added to the end of BlockBasedTable::DumpDataBlocks, so it appears after the data block details, at the very end of the dump file. Test Plan: ``` ./db_bench --benchmarks=fillrandom ./sst_dump --file=/tmp/rocksdbtest-xyz/dbbench/000007.sst --command=raw tail -n 6 /tmp/rocksdbtest-xyz/dbbench/000007_dump.txt ``` ``` Data Block Summary: -------------------------------------- # data blocks: 11336 min data block size: 903 max data block size: 2268 avg data block size: 2245.363356 ``` Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61815
-
由 Aaron Gao 提交于
Summary: preparation for detecting Cache type. If SimCache, we then may trigger some command like "setSimCapacity()" with setOptions() Test Plan: make all check Reviewers: yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61953
-
- 12 8月, 2016 10 次提交
-
-
由 Islam AbdelRahman 提交于
-
由 Islam AbdelRahman 提交于
Summary: This diff update ForwardIterator to support pinning keys and values, which will allow DBIter to take advantage of that and eliminate memcpy when executing merge operators This diff is stacked on D61305 Test Plan: existing tests (updated them to test tailing iterator) new test Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60009
-
由 Adam Retter 提交于
-
由 Adam Retter 提交于
-
由 Dhruba Borthakur 提交于
Test Plan: make check Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61875
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff improves the documentation for GetOptionsFromMap APIs and fixes a bug in GetOptionsFromMap functions in convenience.h where new_options will still be changed when the function call is not successful. Test Plan: options_test Reviewers: IslamAbdelRahman, kradhakrishnan, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61731
-
由 Aaron Gao 提交于
Summary: make transactionDB working with StackableDB Test Plan: make all check -j64 Reviewers: andrewkr, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60705
-
由 krad 提交于
Summary: Travis times out with persistent_cache_test, there is no way to increase the timeout, so lower the test case for Travis Test Plan: Run on travis and sandcastle Subscribers: leveldb, dhruba, andrewkr Differential Revision: https://reviews.facebook.net/D61863
-
由 Islam AbdelRahman 提交于
Summary: This diff include these simple change - Rename ReleasePinnedIterators to ReleasePinnedData - Rename PinIteratorIfNeeded to PinIterator - Use std::vector directly in PinnedIteratorsManager instead of std::unique_ptr<std::vector> - Generalize PinnedIteratorsManager by adding PinPtr which can pin any pointer Test Plan: existing tests Reviewers: sdong, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61305
-
由 Adam Retter 提交于
* Update arc config for ArcanistBaseUnitTestEngine -> ArcanistUnitTestEngine Test Plan: Execute Java test suite Reviewers: yhchiang Differential Revision: https://reviews.facebook.net/D61911 * Fix for arc use of base64 command on Max OS X Test Plan: Run on `arc diff` on Mac OS X and Linux Reviewers: yhchiang Differential Revision: https://reviews.facebook.net/D61917
-