Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 27 3月, 2019 2 次提交
  3. 26 3月, 2019 3 次提交
  5. 22 3月, 2019 3 次提交
    • R
      Make it easier for users to load options from option file and set shared block cache. (#5063) · a4396f92
      由 Rashmi Sharma 提交于 
      Summary:
[RocksDB] Make it easier for users to load options from option file and set shared block cache.
Right now, it requires several dynamic casting for users to set the shared block cache to their option struct cast from the option file.
If people don't do that, every CF of every DB will generate its own 8MB block cache. It's not a usable setting. So we are dragging every user who loads options from the file into such a mess.
Instead, we should allow them to pass their cache object to LoadLatestOptions() and LoadOptionsFromFile(), so that those loaded option structs will have the shared block cache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5063

Differential Revision: D14518584

Pulled By: rashmishrm

fbshipit-source-id: c91430ff9425a0e67d76fc67931d755f491ca5aa
      a4396f92
    • B
      fix NowNanos overflow (#5062) · 88d85b68
      由 Burton Li 提交于 
      Summary:
The original implementation of WinEnvIO::NowNanos() has a constant data overflow by:
li.QuadPart *= std::nano::den;
As a result, the api provides a incorrect result.
e.g.:
li.QuadPart=13477844301545
std::nano::den=1e9

The fix uses pre-computed nano_seconds_per_period_ to present the nano seconds per performance counter period, in the case if nano::den is divisible by perf_counter_frequency_. Otherwise it falls back to use high_resolution_clock.
siying ajkr
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5062

Differential Revision: D14426842

Pulled By: anand1976

fbshipit-source-id: 127f1daf423dd4b30edd0dcf8ea0466f468bec12
      88d85b68
    • M
      Reorder DBIter fields to reduce memory usage (#5078) · c84fad7a
      由 Maysam Yabandeh 提交于 
      Summary:
The patch reorders DBIter fields to put 1-byte fields together and let the compiler optimize the memory usage by using less 64-bit allocations for bools and enums.

This might have a negative side effect of putting the variables that are accessed together into different cache lines and hence increasing the cache misses. Not sure what benchmark would verify that thought. I ran simple, single-threaded seekrandom benchmarks but the variance in the results is too much to be conclusive.

./db_bench --benchmarks=fillrandom --use_existing_db=0 --num=1000000 --db=/dev/shm/dbbench
./db_bench --benchmarks=seekrandom[X10] --use_existing_db=1 --db=/dev/shm/dbbench --num=1000000 --duration=60 --seek_nexts=100
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5078

Differential Revision: D14562676

Pulled By: maysamyabandeh

fbshipit-source-id: 2284655d46e079b6e9a860e94be5defb6f482167
      c84fad7a
  7. 21 3月, 2019 3 次提交
  9. 20 3月, 2019 4 次提交
  11. 19 3月, 2019 2 次提交
    • S
      Feature for sampling and reporting compressibility (#4842) · b45b1cde
      由 Shobhit Dayal 提交于 
      Summary:
This is a feature to sample data-block compressibility and and report them as stats. 1 in N (tunable) blocks is sampled for compressibility using two algorithms:
1. lz4 or snappy for fast compression
2. zstd or zlib for slow but higher compression.

The stats are reported to the caller as raw-bytes and compressed-bytes. The block continues to be compressed for storage using the specified CompressionType.

The db_bench_tool how has a command line option for specifying the sampling rate. It's default value is 0 (no sampling). To test the overhead for a certain value, users can compare the performance of db_bench_tool, varying the sampling rate. It is unlikely to have a noticeable impact for high values like 20.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4842

Differential Revision: D13629011

Pulled By: shobhitdayal

fbshipit-source-id: 14ca668bcab6499b2a1734edf848eb62a4f4fafa
      b45b1cde
    • H
      utilities: Fix build failure with -Werror=maybe-uninitialized (#5074) · 20d49da9
      由 He Zhe 提交于 
      Summary:
Initialize magic_number to zero to avoid such failure.
utilities/blob_db/blob_log_format.cc:91:3: error: 'magic_number' may be used
uninitialized in this function [-Werror=maybe-uninitialized]
   if (magic_number != kMagicNumber) {
   ^~
Signed-off-by: NHe Zhe <zhe.he@windriver.com>
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5074

Differential Revision: D14505514

Pulled By: miasantreble

fbshipit-source-id: 4334462958c2b9c5a7c68c6ab24dadf94ad70902
      20d49da9
  13. 16 3月, 2019 2 次提交
    • A
      Update bg_error when log flush fails in SwitchMemtable() (#5072) · b4fa51df
      由 anand76 提交于 
      Summary:
There is a potential failure case in DBImpl::SwitchMemtable() that is not handled properly. The call to cur_log_writer->WriteBuffer() can fail due to an IO error. In that case, we need to call SetBGError() in order set the background error since the WriteBuffer() failure may result in data loss.

Also, the asserts for !new_mem and !new_log are incorrect, as those would have been allocated by the time this failure is detected.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5072

Differential Revision: D14461384

Pulled By: anand1976

fbshipit-source-id: fb59bce9d61378f37d2dfcd28c0b704b0f43c3cf
      b4fa51df
    • A
      exercise WAL recycling in crash test (#5070) · 2263f869
      由 Andrew Kryczka 提交于 
      Summary:
Since this feature affects the WAL behavior, it seems important our crash-recovery tests cover it.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5070

Differential Revision: D14470085

Pulled By: miasantreble

fbshipit-source-id: 9b9682a718a926d57d055e0a5ec867efbd2eb9c1
      2263f869
  15. 15 3月, 2019 1 次提交
    • Z
      Add the -try_process_corrupted_trace option to trace_analyzer (#5067) · dcde292c
      由 Zhichao Cao 提交于 
      Summary:
In the current trace_analyzer implementation, once the trace file has corrupted content, which can be caused by unexpected tracing operations or other reasons, trace_analyzer will print the error and stop analyzing.

By adding the -try_process_corrupted_trace option, user can try to process the corrupted trace file and get the analyzing results of the trace records from the beginning to the the first corrupted point in the trace file. Analyzing might fail even this option is enabled.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5067

Differential Revision: D14433037

Pulled By: zhichao-cao

fbshipit-source-id: d095233ba371726869af0def0cdee23b69896831
      dcde292c
  17. 13 3月, 2019 2 次提交
  19. 09 3月, 2019 5 次提交
  21. 08 3月, 2019 1 次提交
  23. 07 3月, 2019 2 次提交
    • M
      WritePrepared: handle adding prepare before max_evicted_seq_ (#5025) · 04a2631d
      由 Maysam Yabandeh 提交于 
      Summary:
The patch fixes an improbable race condition between AddPrepared from one write queue and AdvanceMaxEvictedSeq from another queue. In this scenario AddPrepared finds prepare_seq lower than max and adding to PrepareHeap as usual while AdvanceMaxEvictedSeq has finished checking PrepareHeap against the future max. Thus when AdvanceMaxEvictedSeq finishes off by updating the max_evicted_seq_, PrepareHeap ends up with a prepared_seq lower than it which breaks the PrepareHeap contract. The fix is that in AddPrepared we check against the future_max_evicted_seq_ instead, which is update before AdvanceMaxEvictedSeq acquire prepare_mutex_ and looks into PrepareHeap.
A unit test added to test for the failure scenario. The code is also refactored a bit to remove the duplicate code between AdvanceMaxEvictedSeq and AddPrepared.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5025

Differential Revision: D14249028

Pulled By: maysamyabandeh

fbshipit-source-id: 072ea56663f40359662c05fafa6ac524417b0622
      04a2631d
    • M
      WritePrepared: Add rollback batch to PreparedHeap (#5026) · 703f1375
      由 Maysam Yabandeh 提交于 
      Summary:
The patch adds the sequence number of the rollback patch to the PrepareHeap when two_write_queues is enabled. Although the current behavior is still correct, the change simplifies reasoning about the code, by having all uncommitted batches registered with the PreparedHeap.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5026

Differential Revision: D14249401

Pulled By: maysamyabandeh

fbshipit-source-id: 1e3424edee5cd14e56ee35931ad3c93ed997cd5a
      703f1375
  25. 05 3月, 2019 2 次提交
    • A
      Use `fallocate` even if hole-punching unsupported (#5023) · 186b3afa
      由 Andrew Kryczka 提交于 
      Summary:
The compiler flag `-DROCKSDB_FALLOCATE_PRESENT` was only set when
`fallocate`, `FALLOC_FL_KEEP_SIZE`, and `FALLOC_FL_PUNCH_HOLE` were all
present. However, the last of the three is not really necessary for the
primary `fallocate` use case; furthermore, it was introduced only in later
Linux kernel versions (2.6.38+).

This PR changes the flag `-DROCKSDB_FALLOCATE_PRESENT` to only require
`fallocate` and `FALLOC_FL_KEEP_SIZE` to be present. There is a separate
check for `FALLOC_FL_PUNCH_HOLE` only in the place where it is used.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5023

Differential Revision: D14248487

Pulled By: siying

fbshipit-source-id: a10ed0b902fa755988e957bd2dcec9081ec0502e
      186b3afa
    • S
      Move some RocksObject into try-with-resources in Test (#5037) · a2838006
      由 SeterKwok 提交于 
      Summary:
Fix #5008
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5037

Differential Revision: D14302474

Pulled By: riversand963

fbshipit-source-id: dcd9dda5d4d6d459315692f355499a39e546d518
      a2838006
  27. 02 3月, 2019 6 次提交
  29. 01 3月, 2019 2 次提交
    • M
      Call PreReleaseCallback between WAL and memtable write (#5015) · 77ebc82b
      由 Maysam Yabandeh 提交于 
      Summary:
PreReleaseCallback meant to be called before the writes are visible to the readers. Since the sequence number is known after the WAL write, there is no reason to delay calling PreReleaseCallback to after the memtable write, which would complicates the reader's logic in presence of our memtable writes that are made visible by the other write thread.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5015

Differential Revision: D14221670

Pulled By: maysamyabandeh

fbshipit-source-id: a504dd665cf923226d7af09cc8e9c7739a25edc6
      77ebc82b
    • M
      WritePrepared: commit only from the 2nd queue (#5014) · 68a2f94d
      由 Maysam Yabandeh 提交于 
      Summary:
When two_write_queues is enabled we call ::AddPrepared only from the main queue, which writes to both WAL and memtable, and call ::AddCommitted from the 2nd queue, which writes only to WAL. This simplifies the logic by avoiding concurrency between AddPrepared and also between AddCommitted. The patch fixes one case that did not conform with the rule above. This would allow future refactoring. For example AdvaneMaxEvictedSeq, which is invoked by AddCommitted, can be simplified by assuming lack of concurrent calls to it.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5014

Differential Revision: D14210493

Pulled By: maysamyabandeh

fbshipit-source-id: 6db5ba372a294a568a14caa010576460917a4eab
      68a2f94d