- 03 10月, 2017 2 次提交
-
-
由 Adam Retter 提交于
Summary: This enables us to crossbuild pcc64le RocksJava binaries with a suitably old version of glibc (2.17) on CentOS 7. Closes https://github.com/facebook/rocksdb/pull/2491 Differential Revision: D5955301 Pulled By: sagar0 fbshipit-source-id: 69ef9746f1dc30ffde4063dc764583d8c7ae937e
-
由 Siying Dong 提交于
Summary: Make "ldb dump --count_only" print histogram of value size. Also, fix a bug that "ldb dump --path=<db_path>" doesn't work. Closes https://github.com/facebook/rocksdb/pull/2944 Differential Revision: D5954527 Pulled By: siying fbshipit-source-id: c620a444ec544258b8d113f5f663c375dd53d6be
-
- 30 9月, 2017 2 次提交
-
-
由 Zhongyi Xie 提交于
Summary: In SST files, restart interval helps us search in data blocks. However, some meta blocks will be read sequentially, so there's no need for restart points. Restart interval will introduce extra space in the block (https://github.com/facebook/rocksdb/blob/master/table/block_builder.cc#L80). We will see if we can remove this redundant space. (Maybe set restart interval to infinite.) Closes https://github.com/facebook/rocksdb/pull/2940 Differential Revision: D5930139 Pulled By: miasantreble fbshipit-source-id: 92b1b23c15cffa90378343ac846b713623b19c21
-
由 Maysam Yabandeh 提交于
Summary: This template reminds the users to use issues only for bug reports. The template is written according to the github guidelines at https://help.github.com/articles/creating-an-issue-template-for-your-repository/ Closes https://github.com/facebook/rocksdb/pull/2948 Differential Revision: D5943558 Pulled By: maysamyabandeh fbshipit-source-id: c83b5d211ea8e334107141967689b2f0c453bbc9
-
- 29 9月, 2017 7 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: When using with compressed cache it is possible that the status is ok but the block is not actually added to the block cache. The patch takes this case into account. Closes https://github.com/facebook/rocksdb/pull/2945 Differential Revision: D5937613 Pulled By: maysamyabandeh fbshipit-source-id: 5428cf1115e5046b3d01ab78d26cb181122af4c6
-
由 Andrew Kryczka 提交于
Summary: It was broken when `NotifyCollectTableCollectorsOnFinish` was introduced. That function called `Finish` on each of the `TablePropertiesCollector`s, and `CompactOnDeletionCollector::Finish()` was resetting all its internal state. Then, when we checked whether compaction is necessary, the flag had already been cleared. Fixed above issue by avoiding resetting internal state during `Finish()`. Multiple calls to `Finish()` are allowed, but callers cannot invoke `AddUserKey()` on the collector after any finishes. Closes https://github.com/facebook/rocksdb/pull/2936 Differential Revision: D5918659 Pulled By: ajkr fbshipit-source-id: 4f05e9d80e50ee762ba1e611d8d22620029dca6b
-
由 Maysam Yabandeh 提交于
Summary: Recover txns from the WAL. Also added some unit tests. Closes https://github.com/facebook/rocksdb/pull/2901 Differential Revision: D5859596 Pulled By: maysamyabandeh fbshipit-source-id: 6424967b231388093b4effffe0a3b1b7ec8caeb0
-
由 Yu Shu 提交于
Summary: The default one will try to install rocksdb:x86-windows, which would lead to failing of the build at the last step (CMake Error, Rocksdb only supports x64). Because it will try to install a serials of x86 version package, and those cannot proceed to rocksdb:x86-windows building. By using rocksdb:x64-windows, we can make sure to install x64 version. Tested on Win10 x64. Closes https://github.com/facebook/rocksdb/pull/2941 Differential Revision: D5937139 Pulled By: sagar0 fbshipit-source-id: 15637fe23df59326a0e607bd4d5c48733e20bae3
-
由 Sagar Vemuri 提交于
Summary: For every merge operand encountered for a key in the read path we now have the ability to decide whether to look further (to retrieve more merge operands for the key) or stop and invoke the merge operator to return the value. The user needs to override `ShouldMerge()` method with a condition to terminate search when true to avail this facility. This has a couple of advantages: 1. It helps in limiting the number of merge operands that are looked at to compute a value as part of a user Get operation. 2. It allows to peek at a merge key-value to see if further merge operands need to look at. Example: Limiting the number of merge operands that are looked at: Lets say you have 10 merge operands for a key spread over various levels. If you only want RocksDB to look at the latest two merge operands instead of all 10 to compute the value, it is now possible with this PR. You can set the condition in `ShouldMerge()` to return true when the size of the operand list is 2. Look at the example implementation in the unit test. Without this PR, a Get might look at all the 10 merge operands in different levels before invoking the merge-operator. Added a new unit test. Made sure that there is no perf regression by running benchmarks. Command line to Load data: ``` TEST_TMPDIR=/dev/shm ./db_bench --benchmarks="mergerandom" --merge_operator="uint64add" --num=10000000 ... mergerandom : 12.861 micros/op 77757 ops/sec; 8.6 MB/s ( updates:10000000) ``` **ReadRandomMergeRandom bechmark results:** Command line: ``` TEST_TMPDIR=/dev/shm ./db_bench --benchmarks="readrandommergerandom" --merge_operator="uint64add" --num=10000000 ``` Base -- Without this code change (on commit fc7476be): ``` readrandommergerandom : 38.586 micros/op 25916 ops/sec; (reads:3001599 merges:6998401 total:10000000 hits:842235 maxlength:8) ``` With this code change: ``` readrandommergerandom : 38.653 micros/op 25870 ops/sec; (reads:3001599 merges:6998401 total:10000000 hits:842235 maxlength:8) ``` Closes https://github.com/facebook/rocksdb/pull/2923 Differential Revision: D5898239 Pulled By: sagar0 fbshipit-source-id: daefa325019f77968639a75c851d46352c2303ef
-
由 Aliaksei Sandryhaila 提交于
Summary: There is no need for smart pointers in cf_info_map, so use RAII. This should also placate valgrind. Closes https://github.com/facebook/rocksdb/pull/2943 Differential Revision: D5932941 Pulled By: asandryh fbshipit-source-id: 2c37df88573a9df2557880a31193926e4425e054
-
由 Maysam Yabandeh 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2942 Differential Revision: D5932858 Pulled By: maysamyabandeh fbshipit-source-id: e11f52a0b08d65149bb49d99d1dbc82cb5a96fa0
-
- 28 9月, 2017 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: A user encountered segfault on the call to `CacheDependencies()`, probably because `NewIndexIterator()` failed before populating `*index_entry`. Let's avoid the call in that case. Closes https://github.com/facebook/rocksdb/pull/2939 Differential Revision: D5928611 Pulled By: ajkr fbshipit-source-id: 484be453dbb00e5e160e9c6a1bc933df7d80f574
-
由 Quinn Jarrell 提交于
Summary: SUMMARY Moves the bytes_per_sync and wal_bytes_per_sync options from immutableoptions to mutable options. Also if wal_bytes_per_sync is changed, the wal file and memtables are flushed. TEST PLAN ran make check all passed Two new tests SetBytesPerSync, SetWalBytesPerSync check that after issuing setoptions with a new value for the var, the db options have the new value. Closes https://github.com/facebook/rocksdb/pull/2893 Reviewed By: yiwu-arbug Differential Revision: D5845814 Pulled By: TheRushingWookie fbshipit-source-id: 93b52d779ce623691b546679dcd984a06d2ad1bd
-
由 Yi Wu 提交于
Summary: Looks like the API is simply missing. Adding it. Closes https://github.com/facebook/rocksdb/pull/2937 Differential Revision: D5919955 Pulled By: yiwu-arbug fbshipit-source-id: 6e2e9c96c29882b0bb4113d1f8efb72bffc57878
-
- 27 9月, 2017 2 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Expose DestroyDB API in RocksJava. Closes https://github.com/facebook/rocksdb/pull/2934 Differential Revision: D5914775 Pulled By: sagar0 fbshipit-source-id: 84af6ea0d2bccdcfb9fe8c07b2f87373f0d5bab6
-
由 Maysam Yabandeh 提交于
Summary: Its timing out under tsan. Closes https://github.com/facebook/rocksdb/pull/2928 Differential Revision: D5911766 Pulled By: maysamyabandeh fbshipit-source-id: 2faacc07752ac8713a3a2abb5a4c4b7ae3bdf208
-
- 26 9月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2927 Differential Revision: D5906613 Pulled By: siying fbshipit-source-id: 607401e05b27508c816c700864fe81514606e4ef
-
- 23 9月, 2017 3 次提交
-
-
由 Zhongyi Xie 提交于
Summary: Context/problem: - CFs may be flushed at different times - A WAL can only be deleted after all CFs have flushed beyond end of that WAL. - Point-in-time recovery might stop upon reaching the first corruption. - Some CFs may have already flushed beyond that point, while others haven't. We should fail the Open() instead of proceeding with inconsistent CFs. Closes https://github.com/facebook/rocksdb/pull/2900 Differential Revision: D5863281 Pulled By: miasantreble fbshipit-source-id: 180dbaf83d96c804cff49b3c406312a4ae61313e
-
由 Yi Wu 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2922 Differential Revision: D5895310 Pulled By: yiwu-arbug fbshipit-source-id: 52c635a25d22478ec1eca49b6817551202babac2
-
由 Andrew Kryczka 提交于
Summary: Problem: - `DB::SanitizeOptions` strips trailing slash from `wal_dir` but not `dbname` - We check whether `wal_dir` and `dbname` refer to the same directory using string equality: https://github.com/facebook/rocksdb/blob/master/db/repair.cc#L258 - Providing `dbname` with trailing slash causes default `wal_dir` to be misidentified as a separate directory. - Then the repair tries to add all SST files to the `VersionEdit` twice (once for `dbname` dir, once for `wal_dir`) and fails with coredump. Solution: - Add a new `Env` function, `AreFilesSame`, which uses device and inode number to check whether files are the same. It's currently only implemented in `PosixEnv`. - Migrate repair to use `AreFilesSame` to check whether `dbname` and `wal_dir` are same. If unsupported, falls back to string comparison. Closes https://github.com/facebook/rocksdb/pull/2827 Differential Revision: D5761349 Pulled By: ajkr fbshipit-source-id: c839d548678b742af1166d60b09abd94e5476238
-
- 22 9月, 2017 6 次提交
-
-
由 Andrew Kryczka 提交于
Summary: fixes #2902 Closes https://github.com/facebook/rocksdb/pull/2917 Differential Revision: D5887175 Pulled By: ajkr fbshipit-source-id: 364e292c636a3238bfc53b0fb9a01ff2f82dcbb9
-
由 Sagar Vemuri 提交于
Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
-
由 Sagar Vemuri 提交于
Summary: Problem: During RocksJava performance testing we found that the rocksdb jni library is not built with jemalloc; instead it was getting built with the default glibc malloc. We saw quite a bit of memory bloat due to this. Addressed this by installing jemalloc-devel package in the vm that we use to build release jars. Closes https://github.com/facebook/rocksdb/pull/2916 Differential Revision: D5887018 Pulled By: sagar0 fbshipit-source-id: ace0b5d60234b3a30dcd5d39633e7827a5982a50
-
由 PhaniShekhar 提交于
Summary: Fix for [2461](https://github.com/facebook/rocksdb/issues/2461). Problem: When using multiple db_paths setting with RocksDB, RocksDB incorrectly calculates the size of L1 in LevelCompactionBuilder::GetPathId. max_bytes_for_level_base is used as L0 size and L1 size is calculated as (L0 size * max_bytes_for_level_multiplier). However, L1 size should be max_bytes_for_level_base. Solution: Use max_bytes_for_level_base as L1 size. Also, use L1 size as the estimated size of L0. Closes https://github.com/facebook/rocksdb/pull/2903 Differential Revision: D5885442 Pulled By: maysamyabandeh fbshipit-source-id: 036da1c9298d173b9b80479cc6661ee4b7a951f6
-
由 Andrew Kryczka 提交于
Summary: it's unsupported in options file, so the flag should be respected by db_bench even when an options file is provided. Closes https://github.com/facebook/rocksdb/pull/2910 Differential Revision: D5869836 Pulled By: ajkr fbshipit-source-id: f67f591ae083e95e989f86b6fad50765d2e3d855
-
由 Yi Wu 提交于
Summary: The test didn't delete txn before creating a new one. Closes https://github.com/facebook/rocksdb/pull/2913 Differential Revision: D5880236 Pulled By: yiwu-arbug fbshipit-source-id: 7a4fcaada3d86332292754502cd8f4341143bf4f
-
- 21 9月, 2017 1 次提交
-
-
由 Sagar Vemuri 提交于
Summary: This option was introduced in the C++ API in RocksDB 5.6 in bb01c188 . Now, exposing it through RocksJava API. Closes https://github.com/facebook/rocksdb/pull/2908 Differential Revision: D5864224 Pulled By: sagar0 fbshipit-source-id: 140aa55dcf74b14e4d11219d996735c7fdddf513
-
- 20 9月, 2017 3 次提交
-
-
由 Yao Zongyou 提交于
Summary: Hello, current master branch declares cmake_minimum_required (VERSION 2.8.11) but cmake gives the following error: [ 6%] CMake Error at CMakeLists.txt:658 (install): install TARGETS given unknown argument "INCLUDES". CMake Error at src/CMakeLists.txt:658 (install): install TARGETS given unknown argument "INCLUDES". because this argument not supported on CMake versions prior 2.8.12 Closes https://github.com/facebook/rocksdb/pull/2904 Differential Revision: D5863430 Pulled By: yiwu-arbug fbshipit-source-id: 0f7230e080add472ad4b87836b3104ea0b971a38
-
由 Yi Wu 提交于
Summary: This is caught when I was testing #2886. Closes https://github.com/facebook/rocksdb/pull/2907 Differential Revision: D5863153 Pulled By: yiwu-arbug fbshipit-source-id: 8c54759ba1a0dc101f24ab50423e35731300612d
-
由 Orgad Shaneh 提交于
Summary: snprintf is defined as _snprintf, which doesn't exist in the std namespace. Closes https://github.com/facebook/rocksdb/pull/2298 Differential Revision: D5070457 Pulled By: yiwu-arbug fbshipit-source-id: 6e1659ac3e86170653b174578da5a8ed16812cbb
-
- 19 9月, 2017 2 次提交
-
-
由 Pengchao Wang 提交于
Summary: In our testing cluster, we found large amount tombstone has been promoted to kValue type from kMerge after reaching the top level of compaction. Since we used to only collecting tombstone in merge operator, those tombstones can never be collected. This PR addresses the issue by adding a GC step in compaction filter, which is only for kValue type records. Since those record already reached the top of compaction (no earlier data exists) we can safely remove them in compaction filter without worrying old data appears. This PR also removes an old optimization in cassandra merge operator for single merge operands. We need to do GC even on a single operand, so the optimation does not make sense anymore. Closes https://github.com/facebook/rocksdb/pull/2855 Reviewed By: sagar0 Differential Revision: D5806445 Pulled By: wpc fbshipit-source-id: 6eb25629d4ce917eb5e8b489f64a6aa78c7d270b
-
由 Maysam Yabandeh 提交于
Summary: By default the seq number in DB is increased once per written key. WritePrepared txns requires the seq to be increased once per the entire batch so that the seq would be used as the prepare timestamp by which the transaction is identified. Also we need to increase seq for the commit marker since it would give a unique id to the commit timestamp of transactions. Two unit tests are added to verify our understanding of how the seq should be increased. The recovery path requires much more work and is left to another patch. Closes https://github.com/facebook/rocksdb/pull/2885 Differential Revision: D5837843 Pulled By: maysamyabandeh fbshipit-source-id: a08960b93d727e1cf438c254d0c2636fb133cc1c
-
- 16 9月, 2017 4 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Our current implementation of (semi-)copy constructor of DBOptions and ColumnFamilyOptions seems to intend value by value copy, which is what the default copy constructor does anyway. Moreover not using the default constructor has the risk of forgetting to add newly added options. As an example, allow_2pc seems to be forgotten in the copy constructor which was causing one of the unit tests not seeing its effect. Closes https://github.com/facebook/rocksdb/pull/2888 Differential Revision: D5846368 Pulled By: maysamyabandeh fbshipit-source-id: 1ee92a2aeae93886754b7bc039c3411ea2458683
-
由 Siying Dong 提交于
Summary: After 7f6c02dd, the same get_perf_context() is called both of internally and externally. However, I found internally this is not got inlined. I don't know why this is the case, but directly referencing perf_context is the logical way to do. Closes https://github.com/facebook/rocksdb/pull/2892 Differential Revision: D5843789 Pulled By: siying fbshipit-source-id: b49777d8809f35847699291bb7f8ea2754c3af49
-
由 Yi Wu 提交于
Summary: Access of `cfd->current()` needs to hold db mutex. The data race is caught by TSAN but hard to reproduce: https://gist.github.com/yiwu-arbug/0fc6dc0de915297a1740aa9610be9373 Closes https://github.com/facebook/rocksdb/pull/2894 Differential Revision: D5843884 Pulled By: yiwu-arbug fbshipit-source-id: 0a30a421bc96f51840821538ad6453dc0815a942
-
由 Yi Wu 提交于
Summary: Fixing asan error. Closes https://github.com/facebook/rocksdb/pull/2887 Differential Revision: D5838895 Pulled By: yiwu-arbug fbshipit-source-id: 1662ce9856eb5e6877675347dc2240f2acb6fae8
-
- 15 9月, 2017 4 次提交
-
-
由 Ben Clay 提交于
Summary: Plumbed ReadOptions::iterate_upper_bound through JNI. Made the following design choices: * Used Slice instead of AbstractSlice due to the anticipated usecase (key / key prefix). Can change this if anyone disagrees. * Used Slice instead of raw byte[] which seemed cleaner but necessitated the package-private handle-based Slice constructor. Followed WriteBatch as an example. * We need a copy constructor for ReadOptions, as we create one base ReadOptions for a particular usecase and clone -> change the iterate_upper_bound on each slice operation. Shallow copy seemed cleanest. * Hold a reference to the upper bound slice on ReadOptions, in contrast to Snapshot. Signed a Facebook CLA this morning. Closes https://github.com/facebook/rocksdb/pull/2872 Differential Revision: D5824446 Pulled By: sagar0 fbshipit-source-id: 74fc51313a10a81ecd348625e2a50ca5b7766888
-
由 Siying Dong 提交于
Summary: Three small optimizations: (1) iter_->IsKeyPinned() shouldn't be called if read_options.pin_data is not true. This may trigger function call all the way down the iterator tree. (2) reuse the iterator key object in DBIter::FindNextUserEntryInternal(). The constructor of the class has some overheads. (3) Move the switching direction logic in MergingIterator::Next() to a separate function. These three in total improves readseq performance by about 3% in my benchmark setting. Closes https://github.com/facebook/rocksdb/pull/2880 Differential Revision: D5829252 Pulled By: siying fbshipit-source-id: 991aea10c6d6c3b43769cb4db168db62954ad1e3
-
由 Siying Dong 提交于
Summary: Move uncommon code paths in RangeDelAggregator::ShouldDelete() and IterKey::EnlargeBufferIfNeeded() to a separate function, so that the inlined strcuture can be more optimized. Optimize it because these places show up in CPU profiling, though minimum. The performance is really hard measure. I ran db_bench with readseq benchmark against in-memory DB many times. The variation is big, but it seems to show 1% improvements. Closes https://github.com/facebook/rocksdb/pull/2877 Differential Revision: D5828123 Pulled By: siying fbshipit-source-id: 41a49e229f91e9f8409f85cc6f0dc70e31334e4b
-
由 Oleksandr Anyshchenko 提交于
Summary: Added possibility to set save points in transactions and then rollback to them Closes https://github.com/facebook/rocksdb/pull/2876 Differential Revision: D5825829 Pulled By: yiwu-arbug fbshipit-source-id: 62168992340bbcddecdaea3baa2a678475d1429d
-