Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 24 8月, 2018 1 次提交
  3. 23 8月, 2018 2 次提交
  5. 22 8月, 2018 4 次提交
  7. 21 8月, 2018 5 次提交
  9. 18 8月, 2018 3 次提交
    • J
      adds missing PopSavePoint method to Transaction (#4256) · 90f74494
      由 jsteemann 提交于 
      Summary:
Transaction has had methods to deal with SavePoints already, but
was missing the PopSavePoint method provided by WriteBatch and
WriteBatchWithIndex.
This PR adds PopSavePoint to Transaction as well. Having the method
on Transaction-level too is useful for applications that repeatedly
execute a sequence of operations that normally succeed, but infrequently
need to get rolled back. Using SavePoints here is sensible, but as
operations normally succeed the application may pile up a lot of
useless SavePoints inside a Transaction, leading to slightly increased
memory usage for managing the unneeded SavePoints.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4256

Differential Revision: D9326932

Pulled By: yiwu-arbug

fbshipit-source-id: 53a0af18a6c7e87feff8a56f1f3eab9df7f371d6
      90f74494
    • C
      Add CompactRangeOptions for Java (#4220) · c7cf981a
      由 Christian Esken 提交于 
      Summary:
Closes https://github.com/facebook/rocksdb/issues/4195

CompactRangeOptions are available the CPP API, but not in the Java API. This PR adds CompactRangeOptions to the Java API and adds an overloaded compactRange() method. See https://github.com/facebook/rocksdb/issues/4195 for the original discussion.

This change supports all fields of CompactRangeOptions, including the required enum converters in the JNI portal.

Significant changes:
- Make CompactRangeOptions available in the compactRange() for Java.
- Deprecate other compactRange() methods that have individual option params, like in the CPP code.
- Migrate rocksdb_compactrange_helper() to  CompactRangeOptions.
- Add Java unit tests for CompactRangeOptions.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4220

Differential Revision: D9380007

Pulled By: sagar0

fbshipit-source-id: 6af6c334f221427f1997b33fb24c3986b092fed6
      c7cf981a
    • A
      #3865 followup for fix performance degression introduced by switching order of operands (#4284) · fa4de6e3
      由 Andrey Zagrebin 提交于 
      Summary:
Followup for #4266. There is one more place in **get_context.cc** where **MergeOperator::ShouldMerge** should be called with reversed list of operands.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4284

Differential Revision: D9380008

Pulled By: sagar0

fbshipit-source-id: 70ec26e607e5b88465e1acbdcd6c6171bd76b9f2
      fa4de6e3
  11. 17 8月, 2018 4 次提交
  13. 16 8月, 2018 1 次提交
    • F
      Improve point-lookup performance using a data block hash index (#4174) · 19ec44fd
      由 Fenggang Wu 提交于 
      Summary:
Add hash index support to data blocks, which helps to reduce the CPU utilization of point-lookup operations. This feature is backward compatible with the data block created without the hash index. It is disabled by default unless `BlockBasedTableOptions::data_block_index_type` is set to `data_block_index_type = kDataBlockBinaryAndHash.`

The DB size would be bigger with the hash index option as a hash table is added at the end of each data block. If the hash utilization ratio is 1:1, the space overhead is one byte per key. The hash table utilization ratio is adjustable using `BlockBasedTableOptions::data_block_hash_table_util_ratio`. A lower utilization ratio will improve more on the point-lookup efficiency, but take more space too.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4174

Differential Revision: D8965914

Pulled By: fgwu

fbshipit-source-id: 1c6bae5d1fc39c80282d8890a72e9e67bc247198
      19ec44fd
  15. 15 8月, 2018 4 次提交
  17. 14 8月, 2018 4 次提交
    • H
      c-api: add some missing options · d916a110
      由 Huachao Huang 提交于 
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/4267

Differential Revision: D9309505

Pulled By: anand1976

fbshipit-source-id: eb9fee8037f4ff24dc1cdd5cc5ef41c231a03e1f
      d916a110
    • S
      Add a unit test to verify iterators release data blocks after using them (#4170) · f3d91a0b
      由 Siying Dong 提交于 
      Summary:
Add a unit test to check that iterators release data blocks after it has moved away from it. Verify the same for compaction input iterators.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4170

Differential Revision: D8962513

Pulled By: siying

fbshipit-source-id: 05a5b604d7d29887fb488f2cda7286f554a14407
      f3d91a0b
    • Z
      RocksDB Trace Analyzer (#4091) · 999d955e
      由 Zhichao Cao 提交于 
      Summary:
A framework of trace analyzing for RocksDB

After collecting the trace by using the tool of [PR #3837](https://github.com/facebook/rocksdb/pull/3837). User can use the Trace Analyzer to interpret, analyze, and characterize the collected workload.
**Input:**
1. trace file
2. Whole keys space file

**Statistics:**
1. Access count of each operation (Get, Put, Delete, SingleDelete, DeleteRange, Merge) in each column family.
2. Key hotness (access count) of each one
3. Key space separation based on given prefix
4. Key size distribution
5. Value size distribution if appliable
6. Top K accessed keys
7. QPS statistics including the average QPS and peak QPS
8. Top K accessed prefix
9. The query correlation analyzing, output the number of X after Y and the corresponding average time
    intervals

**Output:**
1. key access heat map (either in the accessed key space or whole key space)
2. trace sequence file (interpret the raw trace file to line base text file for future use)
3. Time serial (The key space ID and its access time)
4. Key access count distritbution
5. Key size distribution
6. Value size distribution (in each intervals)
7. whole key space separation by the prefix
8. Accessed key space separation by the prefix
9. QPS of each operation and each column family
10. Top K QPS and their accessed prefix range

**Test:**
1. Added the unit test of analyzing Get, Put, Delete, SingleDelete, DeleteRange, Merge
2. Generated the trace and analyze the trace

**Implemented but not tested (due to the limitation of trace_replay):**
1. Analyzing Iterator, supporting Seek() and SeekForPrev() analyzing
2. Analyzing the number of Key found by Get

**Future Work:**
1.  Support execution time analyzing of each requests
2.  Support cache hit situation and block read situation of Get
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4091

Differential Revision: D9256157

Pulled By: zhichao-cao

fbshipit-source-id: f0ceacb7eedbc43a3eee6e85b76087d7832a8fe6
      999d955e
    • Y
      Remove an assersion about file size (#4268) · 1b1d2643
      由 Yanqin Jin 提交于 
      Summary:
Due to 4ea56b1b, we should also remove the
assersion in stress test. This removal can be temporary, and we can add it back
once we figure out the reason for the 0-byte SSTs.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4268

Differential Revision: D9297186

Pulled By: riversand963

fbshipit-source-id: cebba9a68f42e815f8cf24471176d2cfdf962f63
      1b1d2643
  19. 12 8月, 2018 1 次提交
    • A
      Revert changes in PR #4003 (#4263) · 4ea56b1b
      由 Anand Ananthabhotla 提交于 
      Summary:
Revert this change. Not generating the OnTableFileCreated() notification for a 0 byte SST on flush breaks the assumption that every OnTableFileCreationStarted() notification is followed by a corresponding OnTableFileCreated().
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4263

Differential Revision: D9285623

Pulled By: anand1976

fbshipit-source-id: 808c3dcd498b4b4f4ed4be947a29a24b2296aa8d
      4ea56b1b
  21. 11 8月, 2018 4 次提交
  23. 10 8月, 2018 7 次提交
    • Y
      Fix a TSAN failure (#4250) · b271f956
      由 Yanqin Jin 提交于 
      Summary:
TSAN fails due to comparison between signed int and unsigned long. Fix it by
static_casting.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4250

Differential Revision: D9256535

Pulled By: riversand963

fbshipit-source-id: c6bad23ff70c6d0ec58e2e85c401ce0ad45de609
      b271f956
    • M
      Index value delta encoding (#3983) · caf0f53a
      由 Maysam Yabandeh 提交于 
      Summary:
Given that index value is a BlockHandle, which is basically an <offset, size> pair we can apply delta encoding on the values. The first value at each index restart interval encoded the full BlockHandle but the rest encode only the size. Refer to IndexBlockIter::DecodeCurrentValue for the detail of the encoding. This reduces the index size which helps using the  block cache more efficiently. The feature is enabled with using format_version 4.

The feature comes with a bit of cpu overhead which should be paid back by the higher cache hits due to smaller index block size.
Results with sysbench read-only using 4k blocks and using 16 index restart interval:
Format 2:
19585   rocksdb read-only range=100
Format 3:
19569   rocksdb read-only range=100
Format 4:
19352   rocksdb read-only range=100
Pull Request resolved: https://github.com/facebook/rocksdb/pull/3983

Differential Revision: D8361343

Pulled By: maysamyabandeh

fbshipit-source-id: f882ee082322acac32b0072e2bdbb0b5f854e651
      caf0f53a
    • A
      Update FB internal dependencies (#4244) · 63f265c3
      由 Andrew Kryczka 提交于 
      Summary:
I noticed we were building against zstd 1.3.0 which is missing optimizations that our fbcode customers have (they're on zstd 1.3.5). Ran `./build_tools/update_dependencies.sh` to catch us up. Omitted the changes it made for gcc-4.8 since it's broken.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4244

Differential Revision: D9230111

Pulled By: ajkr

fbshipit-source-id: 3e8ec1d8a961f98ec77c8c6580bde4caacf2d437
      63f265c3
    • A
      Fix error parsing in build_tools/error_filter.py (#4247) · 72e69497
      由 Anand Ananthabhotla 提交于 
      Summary:
The error_filter.py script parses the output of the "Build and run" stage of continuous tests to check for errors. It is currently only detecting compile errors and not link errors. This change fixes that.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4247

Differential Revision: D9233735

Pulled By: anand1976

fbshipit-source-id: 16e5a04950891cd9aba5cb3efcb6abc2a2e0d5ae
      72e69497
    • G
      Make rocksdb::Slice more interoperable with std::string_view (#4242) · 1b813a9b
      由 Georgios Bitzes 提交于 
      Summary:
This change allows using std::string_view objects directly in the
DB API:

    db->Get(some_string_view_object, ...);

The conversion from std::string_view to rocksdb::Slice is done
automatically, thanks to the added constructor.

I'm stopping short of adding an implicit conversion operator
from rocksdb::Slice to std::string_view, as I don't think that's
a good idea for PinnableSlices.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4242

Differential Revision: D9224134

Pulled By: anand1976

fbshipit-source-id: f50aad04dd0b01737907c0fb88d495c83a81f4e4
      1b813a9b
    • D
      Implement Env::NumFileLinks (#4221) · ab22cf34
      由 Dmitri Smirnov 提交于 
      Summary:
Although delete scheduler implementation allows for the interface not to be supported, the delete_scheduler_test does not allow for that.
Address compiler warnings
Make sst_dump_test use test directory structure as the current execution directory may not be writiable.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4221

Differential Revision: D9210152

Pulled By: siying

fbshipit-source-id: 381a74511e969ecb8089d5c4b4df87dc30c8df63
      ab22cf34
    • Y
      Add SST ingestion to ldb (#4205) · de7f423a
      由 Yanqin Jin 提交于 
      Summary:
We add two subcommands `write_extern_sst` and `ingest_extern_sst` to ldb. This PR avoids changing existing code because we hope to cherry-pick to earlier releases to support compatibility check for external SST file ingestion.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4205

Differential Revision: D9112711

Pulled By: riversand963

fbshipit-source-id: 7cae88380d4de86da8440230e87eca66755648e4
      de7f423a