Skip to content

R
rocksdb

kvdb / rocksdb
11 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 23 2月, 2017 2 次提交
  3. 22 2月, 2017 4 次提交
    • D
      Page size isn't always 4k on linux · f0879e4c
      由 Daniel Black 提交于 
      Summary:
Some places autodetected. These are the two places that didn't.

closes #1498

Still unsure if the following instances of 4 * 1024 need fixing in:
util/io_posix.h
include/rocksdb/table.h (appears to be blocksize and different)
utilities/persistent_cache/block_cache_tier.cc
utilities/persistent_cache/persistent_cache_test.h
include/rocksdb/env.h
util/env_posix.cc
db/column_family.cc
Closes https://github.com/facebook/rocksdb/pull/1499

Differential Revision: D4593640

Pulled By: yiwu-arbug

fbshipit-source-id: efc48de
      f0879e4c
    • M
      Fix interference between max_total_wal_size and db_write_buffer_size checks · 18eeb7b9
      由 Mike Kolupaev 提交于 
      Summary:
This is a trivial fix for OOMs we've seen a few days ago in logdevice.

RocksDB get into the following state:
(1) Write throughput is too high for flushes to keep up. Compactions are out of the picture - automatic compactions are disabled, and for manual compactions we don't care that much if they fall behind. We write to many CFs, with only a few L0 sst files in each, so compactions are not needed most of the time.
(2) total_log_size_ is consistently greater than GetMaxTotalWalSize(). It doesn't get smaller since flushes are falling ever further behind.
(3) Total size of memtables is way above db_write_buffer_size and keeps growing. But the write_buffer_manager_->ShouldFlush() is not checked because (2) prevents it (for no good reason, afaict; this is what this commit fixes).
(4) Every call to WriteImpl() hits the MaybeFlushColumnFamilies() path. This keeps flushing the memtables one by one in order of increasing log file number.
(5) No write stalling trigger is hit. We rely on max_write_buffer_number
Closes https://github.com/facebook/rocksdb/pull/1893

Differential Revision: D4593590

Pulled By: yiwu-arbug

fbshipit-source-id: af79c5f
      18eeb7b9
    • I
      Temporarly return deprecated functions to fix MongoRocks build · 1560b2f5
      由 Islam AbdelRahman 提交于 
      Summary:
MongoRocks is still using some deprecated functions, return them temporarily
Closes https://github.com/facebook/rocksdb/pull/1892

Differential Revision: D4592451

Pulled By: IslamAbdelRahman

fbshipit-source-id: 5e6be3e
      1560b2f5
    • A
      level compaction expansion · 2a0f3d0d
      由 Aaron Gao 提交于 
      Summary:
reimplement the compaction expansion on lower level.

Considering such a case:
input level file: 1[B E] 2[F G] 3[H I] 4 [J M]
output level file: 5[A C] 6[D K] 7[L O]

If we initially pick file 2, now we will compact file 2 and 6. But we can safely compact 2, 3 and 6 without expanding the output level.

The previous code is messy and wrong.

In this diff, I first determine the input range [a, b], and output range [c, d],
then we get the range [e,f] = [min(a, c), max(b, d] and put all eligible clean-cut files within [e, f] into this compaction.

**Note: clean-cut means the files don't have the same user key on the boundaries of some files that are not chosen in this compaction**.
Closes https://github.com/facebook/rocksdb/pull/1760

Differential Revision: D4395564

Pulled By: lightmark

fbshipit-source-id: 2dc2c5c
      2a0f3d0d
  5. 19 2月, 2017 1 次提交
  7. 18 2月, 2017 4 次提交
  9. 17 2月, 2017 4 次提交
  11. 16 2月, 2017 1 次提交
  13. 15 2月, 2017 1 次提交
  15. 14 2月, 2017 2 次提交
    • Y
      Make DBImpl::has_unpersisted_data_ atomic · c2247dc1
      由 Yi Wu 提交于 
      Summary:
Seems to me `has_unpersisted_data_` is read from read thread and write
from write thread concurrently without synchronization. Making it an
atomic.

I update the logic not because seeing any problem with it, but it just
feel confusing.
Closes https://github.com/facebook/rocksdb/pull/1869

Differential Revision: D4555837

Pulled By: yiwu-arbug

fbshipit-source-id: eff2ab8
      c2247dc1
    • S
      Remove disableDataSync option · eb912a92
      由 Sagar Vemuri 提交于 
      Summary:
Remove disableDataSync, and another similarly named disable_data_sync options.
This is being done to simplify options, and also because the performance gains of this feature can be achieved by other methods.
Closes https://github.com/facebook/rocksdb/pull/1859

Differential Revision: D4541292

Pulled By: sagar0

fbshipit-source-id: 5b3a6ca
      eb912a92
  17. 13 2月, 2017 1 次提交
  19. 11 2月, 2017 4 次提交
  21. 09 2月, 2017 4 次提交
  23. 08 2月, 2017 4 次提交
  25. 07 2月, 2017 3 次提交
    • M
      Two-level Indexes · 69d5262c
      由 Maysam Yabandeh 提交于 
      Summary:
Partition Index blocks and use a Partition-index as a 2nd level index.

The two-level index can be used by setting
BlockBasedTableOptions::kTwoLevelIndexSearch as the index type and
configuring BlockBasedTableOptions::index_per_partition

t15539501
Closes https://github.com/facebook/rocksdb/pull/1814

Differential Revision: D4473535

Pulled By: maysamyabandeh

fbshipit-source-id: bffb87e
      69d5262c
    • D
      Windows thread · 0a4cdde5
      由 Dmitri Smirnov 提交于 
      Summary:
introduce new methods into a public threadpool interface,
- allow submission of std::functions as they allow greater flexibility.
- add Joining methods to the implementation to join scheduled and submitted jobs with
  an option to cancel jobs that did not start executing.
- Remove ugly `#ifdefs` between pthread and std implementation, make it uniform.
- introduce pimpl for a drop in replacement of the implementation
- Introduce rocksdb::port::Thread typedef which is a replacement for std::thread.  On Posix Thread defaults as before std::thread.
- Implement WindowsThread that allocates memory in a more controllable manner than windows std::thread with a replaceable implementation.
- should be no functionality changes.
Closes https://github.com/facebook/rocksdb/pull/1823

Differential Revision: D4492902

Pulled By: siying

fbshipit-source-id: c74cb11
      0a4cdde5
    • V
      Adding GetApproximateMemTableStats method · 1aaa898c
      由 Vitaliy Liptchinsky 提交于 
      Summary:
Added method that returns approx num of entries as well as size for memtables.
Closes https://github.com/facebook/rocksdb/pull/1841

Differential Revision: D4511990

Pulled By: VitaliyLi

fbshipit-source-id: 9a4576e
      1aaa898c
  27. 04 2月, 2017 3 次提交
  29. 03 2月, 2017 2 次提交