Skip to content

R
rocksdb

kvdb / rocksdb
11 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 18 6月, 2014 2 次提交
  3. 17 6月, 2014 4 次提交
    • Y
      Include max_write_buffer_number >= 2 to SanitizeOptions. · e6e259b8
      由 Yueh-Hsuan Chiang 提交于 
      Summary: Include max_write_buffer_number >= 2 to SanitizeOptions.

Test Plan: make all check

Reviewers: haobo, sdong, igor, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19077
      e6e259b8
    • S
      Refactor: group metadata needed to open an SST file to a separate copyable struct · cadc1adf
      由 sdong 提交于 
      Summary:
We added multiple fields to FileMetaData recently and are planning to add more.
This refactoring separate the minimum information for accessing the file. This object is copyable (FileMetaData is not copyable since the ref counter). I hope this refactoring can enable further improvements:

(1) use it to design a more efficient data structure to speed up read queries.
(2) in the future, when we add information of storage level, we can easily do the encoding, instead of enlarge this structure, which might expand memory work set for file meta data.

The definition is same as current EncodedFileMetaData used in two level iterator, so now the logic in two level iterator is easier to understand.

Test Plan: make all check

Reviewers: haobo, igor, ljin

Reviewed By: ljin

Subscribers: leveldb, dhruba, yhchiang

Differential Revision: https://reviews.facebook.net/D18933
      cadc1adf
    • B
      Add separate Read/WriteUnlock methods in MutexRW. · 2d02ec65
      由 Bradley Grainger 提交于 
      Some platforms, particularly Windows, do not have a single method that can
release both a held reader lock and a held writer lock; instead, a
separate method (ReleaseSRWLockShared or ReleaseSRWLockExclusive) must be
called in each case.

This may also be necessary to back MutexRW with a shared_mutex in C++14;
the current language proposal includes both an unlock() and a
shared_unlock() method.
      2d02ec65
    • Y
      Fix a bug causing LOG is not created when max_log_file_size is set. · 4d913cfb
      由 Yueh-Hsuan Chiang 提交于 
      Summary:
Fix a bug causing LOG is not created when max_log_file_size is set.
This bug is reported in issue #174.

Test Plan:
Add TEST(AutoRollLoggerTest, LogFileExistence).
make auto_roll_logger_test
./auto_roll_logger_test

Reviewers: haobo, sdong, ljin, igor, igor2

Reviewed By: igor2

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D19053
      4d913cfb
  5. 14 6月, 2014 2 次提交
  7. 13 6月, 2014 2 次提交
  9. 11 6月, 2014 5 次提交
  11. 10 6月, 2014 3 次提交
    • S
      Clean PlainTableReader's variables for better data locality · 80f409ea
      由 sdong 提交于 
      Summary:
Clean PlainTableReader's data structures:
(1) inline bloom_ (in order to do this, change DynamicBloom to allow lazy initialization)
(2) remove some variables only used when initialization from the class
(3) put variables not used in normal read code paths to the end of the class and reference prefix_extractor directly
(4) make Options a reference.

Test Plan: make all check

Reviewers: haobo, ljin

Reviewed By: ljin

Subscribers: igor, yhchiang, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D18891
      80f409ea
    • I
      Don't compress block bigger than 2GB · f43c8262
      由 Igor Canadi 提交于 
      Summary: This is a temporary solution to a issue that we have with compression libraries. See task #4453446.

Test Plan: make check doesn't complain :)

Reviewers: haobo, ljin, yhchiang, dhruba, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18975
      f43c8262
    • S
      sst_dump: still try to print out table properties even if failing to read the file · ee5a51e6
      由 sdong 提交于 
      Summary: Even if the file is corrupted, table properties are usually available to print out. Now sst_dump would just fail without printing table properties. With this patch, table properties are still try to be printed out.

Test Plan: run sst_dump against multiple scenarios

Reviewers: igor, yhchiang, ljin, haobo

Reviewed By: haobo

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D18981
      ee5a51e6
  13. 08 6月, 2014 1 次提交
  15. 07 6月, 2014 3 次提交
  17. 06 6月, 2014 4 次提交
  19. 05 6月, 2014 4 次提交
  21. 04 6月, 2014 2 次提交
  23. 03 6月, 2014 5 次提交
    • S
      In DB::NewIterator(), try to allocate the whole iterator tree in an arena · df9069d2
      由 sdong 提交于 
      Summary:
In this patch, try to allocate the whole iterator tree starting from DBIter from an arena
1. ArenaWrappedDBIter is created when serves as the entry point of an iterator tree, with an arena in it.
2. Add an option to create iterator from arena for following iterators: DBIter, MergingIterator, MemtableIterator, all mem table's iterators, all table reader's iterators and two level iterator.
3. MergeIteratorBuilder is created to incrementally build the tree of internal iterators. It is passed to mem table list and version set and add iterators to it.

Limitations:
(1) Only DB::NewIterator() without tailing uses the arena. Other cases, including readonly DB and compactions are still from malloc
(2) Two level iterator itself is allocated in arena, but not iterators inside it.

Test Plan: make all check

Reviewers: ljin, haobo

Reviewed By: haobo

Subscribers: leveldb, dhruba, yhchiang, igor

Differential Revision: https://reviews.facebook.net/D18513
      df9069d2
    • S
      dynamic_bloom: replace some divide (remainder) operations with shifts in... · 46279669
      由 sdong 提交于 
      dynamic_bloom: replace some divide (remainder) operations with shifts in locality mode, and other improvements

Summary:
This patch changes meaning of options.bloom_locality: 0 means disable cache line optimization and any positive number means use CACHE_LINE_SIZE as block size (the previous behavior is the block size will be CACHE_LINE_SIZE*options.bloom_locality). By doing it, the divide operations inside a block can be replaced by a shift.

Performance is improved:
https://reviews.facebook.net/P471

Also, improve the basic algorithm in two ways:
(1) make sure num of blocks is an odd number
(2) rotate bytes after every probe in locality mode. Since the divider is 2^n, unless doing it, we are never able to use all the bits.
Improvements of false positive: https://reviews.facebook.net/P459

Test Plan: make all check

Reviewers: ljin, haobo

Reviewed By: haobo

Subscribers: dhruba, yhchiang, igor, leveldb

Differential Revision: https://reviews.facebook.net/D18843
      46279669
    • I
      Only signal cond variable if need to · 91ddd587
      由 Igor Canadi 提交于 
      Summary:
At the end of BackgroundCallCompaction(), we call SignalAll(), even though we don't need to. If compaction hasn't done anything and there's another compaction running, there is no need to signal on the condition variable. Doing so creates a tight feedback loop which results in log files like:

   wait for memtable flush
   compaction nothing to do
   wait for memtable flush
   compaction nothing to do

This change eliminates that

Test Plan:
make check
Also:

    icanadi@dev1440 ~ $ grep "nothing to do" /fast-rocksdb-tmp/rocksdb_test/column_family_test/LOG | wc -l
    7435
    icanadi@dev1440 ~ $ grep "nothing to do" /fast-rocksdb-tmp/rocksdb_test/column_family_test/LOG | wc -l
    372

First version is before the change, second version is after the change.

Reviewers: dhruba, ljin, haobo, yhchiang, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18855
      91ddd587
    • I
      Flush stale column families less aggressively · 8cb7ad83
      由 Igor Canadi 提交于 
      Summary:
We've seen some production issues where column family is detected as stale, although there is only one column family in the system. This is a quick fix that:
1) doesn't flush stale column families if there's only one of them
2) Use 4 as a coefficient instead of 2 for determening when a column family is stale. This will make flushing less aggressive, while still keep a nice dynamic flushing of very stale CFs.

Test Plan: make check

Reviewers: dhruba, haobo, ljin, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18861
      8cb7ad83
    • S
      db_stress to add an option to periodically change background thread pool size. · 593bb2c4
      由 sdong 提交于 
      Summary: Add an option to indicates the variation of background threads. If the flag is not 0, for every 100 milliseconds, adjust thread pool size to a value within the range.

Test Plan: run db_stress

Reviewers: haobo, dhruba, igor, ljin

Reviewed By: ljin

Subscribers: leveldb, nkg-

Differential Revision: https://reviews.facebook.net/D18759
      593bb2c4
  25. 31 5月, 2014 2 次提交
    • L
      forward iterator · 388d2054
      由 Lei Jin 提交于 
      Summary:
Forward iterator puts everything together in a flat structure instead of
a hierarchy of nested iterators. this should simplify the code and
provide better performance. It also enables more optimization since all
information are accessiable in one place.
Init evaluation shows about 6% improvement

Test Plan: db_test and db_bench

Reviewers: dhruba, igor, tnovak, sdong, haobo

Reviewed By: haobo

Subscribers: sdong, leveldb

Differential Revision: https://reviews.facebook.net/D18795
      388d2054
    • L
      add an iterator refresh option for SeekRandom · f29c62fc
      由 Lei Jin 提交于 
      Summary: One more option to allow iterator refreshing when using normal iterator

Test Plan: ran db_bench

Reviewers: haobo, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18849
      f29c62fc
  27. 30 5月, 2014 1 次提交