Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 23 10月, 2018 1 次提交
    • S
      Dynamic level to adjust level multiplier when write is too heavy (#4338) · 70242636
      由 Siying Dong 提交于 
      Summary:
Level compaction usually performs poorly when the writes so heavy that the level targets can't be guaranteed. With this improvement, we improve level_compaction_dynamic_level_bytes = true so that in the write heavy cases, the level multiplier can be slightly adjusted based on the size of L0.

We keep the behavior the same if number of L0 files is under 2X compaction trigger and the total size is less than options.max_bytes_for_level_base, so that unless write is so heavy that compaction cannot keep up, the behavior doesn't change.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4338

Differential Revision: D9636782

Pulled By: siying

fbshipit-source-id: e27fc17a7c29c84b00064cc17536a01dacef7595
      70242636
  3. 22 10月, 2018 1 次提交
    • Y
      Fix RepeatableThreadTest::MockEnvTest hang (#4560) · 933250e3
      由 Yi Wu 提交于 
      Summary:
When `MockTimeEnv` is used in test to mock time methods, we cannot use `CondVar::TimedWait` because it is using real time, not the mocked time for wait timeout. On Mac the method can return immediately without awaking other waiting threads, if the real time is larger than `wait_until` (which is a mocked time). When that happen, the `wait()` method will fall into an infinite loop.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4560

Differential Revision: D10472851

Pulled By: yiwu-arbug

fbshipit-source-id: 898902546ace7db7ac509337dd8677a527209d19
      933250e3
  5. 20 10月, 2018 3 次提交
    • S
      Fix printf formatting on MacOS (#4533) · f959e880
      由 Simon Grätzer 提交于 
      Summary:
On MacOS with clang the compilation of _tools/db_bench_tool.cc_ always fails because the format used in a `fprintf` call has the wrong type. This PR should hopefully fix this issue
```
tools/db_bench_tool.cc:4233:61: error: format specifies type 'unsigned long long' but the argument has type 'size_t' (aka 'unsigned long')
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4533

Differential Revision: D10471657

Pulled By: maysamyabandeh

fbshipit-source-id: f20f5f3756d3571b586c895c845d0d4d1e34a398
      f959e880
    • S
      Fix WriteBatchWithIndex's SeekForPrev() (#4559) · c17383f9
      由 Siying Dong 提交于 
      Summary:
WriteBatchWithIndex's SeekForPrev() has a bug that we internally place the position just before the seek key rather than after. This makes the iterator to miss the result that is the same as the seek key. Fix it by position the iterator equal or smaller.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4559

Differential Revision: D10468534

Pulled By: siying

fbshipit-source-id: 2fb371ae809c561b60a1c11cef71e1c66fea1f19
      c17383f9
    • Y
      Add read retry support to log reader (#4394) · da4aa59b
      由 Yanqin Jin 提交于 
      Summary:
Current `log::Reader` does not perform retry after encountering `EOF`. In the future, we need the log reader to be able to retry tailing the log even after `EOF`.

Current implementation is simple. It does not provide more advanced retry policies. Will address this in the future.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4394

Differential Revision: D9926508

Pulled By: riversand963

fbshipit-source-id: d86d145792a41bd64a72f642a2a08c7b7b5201e1
      da4aa59b
  7. 19 10月, 2018 4 次提交
  9. 18 10月, 2018 3 次提交
    • J
      Plumb WriteBufferManager through JNI (#4492) · a4d9aa6b
      由 Jigar Bhati 提交于 
      Summary:
Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4492

Differential Revision: D10428506

Pulled By: sagar0

fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
      a4d9aa6b
    • A
      Lazily initialize RangeDelAggregator stripe map entries (#4497) · 45f213b5
      由 Abhishek Madan 提交于 
      Summary:
When there are no range deletions, flush and compaction perform a binary search
on an effectively empty map every time they call ShouldDelete. This PR lazily
initializes each stripe map entry so that the binary search can be elided in
these cases.

After this PR, the total amount of time spent in compactions is 52.541331s, and the total amount of time spent in flush is 5.532608s, the former of which is a significant improvement from the results after #4495.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4497

Differential Revision: D10428610

Pulled By: abhimadan

fbshipit-source-id: 6f7e1ce3698fac3ef86d1197955e6b72e0931a0f
      45f213b5
    • Z
      Add PerfContextByLevel to provide per level perf context information (#4226) · d6ec2887
      由 Zhongyi Xie 提交于 
      Summary:
Current implementation of perf context is level agnostic. Making it hard to do performance evaluation for the LSM tree. This PR adds `PerfContextByLevel` to decompose the counters by level.
This will be helpful when analyzing point and range query performance as well as tuning bloom filter
Also replaced __thread with thread_local keyword for perf_context
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4226

Differential Revision: D10369509

Pulled By: miasantreble

fbshipit-source-id: f1ced4e0de5fcebdb7f9cff36164516bc6382d82
      d6ec2887
  11. 16 10月, 2018 6 次提交
    • A
      Properly determine a truncated CompactRange stop key (#4496) · 1e384580
      由 anand1976 提交于 
      Summary:
When a CompactRange() call for a level is truncated before the end key
is reached, because it exceeds max_compaction_bytes, we need to properly
set the compaction_end parameter to indicate the stop key. The next
CompactRange will use that as the begin key. We set it to the smallest
key of the next file in the level after expanding inputs to get a clean
cut.

Previously, we were setting it before expanding inputs. So we could end
up recompacting some files. In a pathological case, where a single key
has many entries spanning all the files in the level (possibly due to
merge operands without a partial merge operator, thus resulting in
compaction output identical to the input), this would result in
an endless loop over the same set of files.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4496

Differential Revision: D10395026

Pulled By: anand1976

fbshipit-source-id: f0c2f89fee29b4b3be53b6467b53abba8e9146a9
      1e384580
    • Y
      Add support to flush multiple CFs atomically (#4262) · e633983c
      由 Yanqin Jin 提交于 
      Summary:
Leverage existing `FlushJob` to implement atomic flush of multiple column families.

This PR depends on other PRs and is a subset of #3752 . This PR itself is not sufficient in fulfilling atomic flush.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4262

Differential Revision: D9283109

Pulled By: riversand963

fbshipit-source-id: 65401f913e4160b0a61c0be6cd02adc15dad28ed
      e633983c
    • A
      Avoid per-key linear scan over snapshots in compaction (#4495) · 32b4d4ad
      由 Andrew Kryczka 提交于 
      Summary:
`CompactionIterator::snapshots_` is ordered by ascending seqnum, just like `DBImpl`'s linked list of snapshots from which it was copied. This PR exploits this ordering to make `findEarliestVisibleSnapshot` do binary search rather than linear scan. This can make flush/compaction significantly faster when many snapshots exist since that function is called on every single key.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4495

Differential Revision: D10386470

Pulled By: ajkr

fbshipit-source-id: 29734991631227b6b7b677e156ac567690118a8b
      32b4d4ad
    • M
      Update WritePrepared blog post with latest results (#4494) · 0f955f2a
      由 Maysam Yabandeh 提交于 
      Summary:
WritePrepared is declared production ready (overdue update) and the benchmark results are also reported.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4494

Differential Revision: D10385336

Pulled By: maysamyabandeh

fbshipit-source-id: 662672ddfa286aa46af544f505b4d4b7a882d408
      0f955f2a
    • Y
      Replace 'string' with 'const string&' in FileOperationInfo (#4491) · ce522746
      由 Yanqin Jin 提交于 
      Summary:
Using const string& can avoid one extra string copy. This PR addresses a recent comment made by siying  on #3933.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4491

Differential Revision: D10381211

Pulled By: riversand963

fbshipit-source-id: 27fc2d65d84bc7cd07833c77cdc47f06dcfaeb31
      ce522746
    • Y
      Set -DROCKSDB_JEMALLOC for buck build if jemalloc presents (#4489) · f60c4e5a
      由 Yi Wu 提交于 
      Summary:
Set the macro if default allocator is jemalloc. It doesn't handle the case when allocator is specified, e.g.
```
cpp_binary(
    name="xxx"
    allocator="jemalloc", # or "malloc" or something else
    deps=["//rocksdb:rocksdb"],
)
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4489

Differential Revision: D10363683

Pulled By: yiwu-arbug

fbshipit-source-id: 5da490336a8e78e0feb0900c29e8036e7ec6f12b
      f60c4e5a
  13. 13 10月, 2018 4 次提交
  15. 12 10月, 2018 3 次提交
    • W
      Add compile time option to work with utf8 filename strings (#4469) · 5d809ece
      由 Wilfried Goesgens 提交于 
      Summary:
The default behaviour of rocksdb is to use the `*A(` windows API functions.
These accept filenames in the currently configured system encoding,
be it Latin 1, utf8 or whatever.
If the Application intends to completely work with utf8 strings internally,
converting these to that codepage properly isn't even always possible.
Thus this patch adds a switch to use the `*W(` functions, which accept
UTF-16 filenames, and uses C++11 features to translate the
UTF8 containing std::string to an UTF16 containing std::wstring.

This feature is a compile time options, that can be enabled by setting `WITH_WINDOWS_UTF8_FILENAMES` to true.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4469

Differential Revision: D10356011

Pulled By: yiwu-arbug

fbshipit-source-id: 27b6ae9171f209085894cdf80069e8a896642044
      5d809ece
    • A
      Use vector in UncollapsedRangeDelMap (#4487) · 7dd16410
      由 Abhishek Madan 提交于 
      Summary:
Using `./range_del_aggregator_bench --use_collapsed=false
--num_range_tombstones=5000 --num_runs=1000`, here are the results before and
after this change:

Before:
```
=========================
Results:
=========================
AddTombstones:           1822.61 us
ShouldDelete (first):    94.5286 us
```

After:
```
=========================
Results:
=========================
AddTombstones:           199.26 us
ShouldDelete (first):    38.9344 us
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4487

Differential Revision: D10347288

Pulled By: abhimadan

fbshipit-source-id: d44efe3a166d583acfdc3ec1199e0892f34dbfb7
      7dd16410
    • Z
      C++17 support (#4482) · 46dd8b1e
      由 zpalmtree 提交于 
      Summary:
Closes https://github.com/facebook/rocksdb/issues/4462

I'm not sure if you'll be happy with `std::random_device{}`, perhaps you would want to use your rand instance instead. I didn't test to see if your rand instance supports the requirements that `std::shuffle` takes.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4482

Differential Revision: D10325133

Pulled By: yiwu-arbug

fbshipit-source-id: 47b7adaf4bb2b8d64cf090ea6b1b48ef53180581
      46dd8b1e
  17. 11 10月, 2018 9 次提交
  19. 10 10月, 2018 6 次提交