Skip to content

R
rocksdb

kvdb / rocksdb
10 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

前往新版Gitcode，体验更适合开发者的 AI 搜索 >>

切换分支/标签
  1. 21 3月, 2017 3 次提交
  3. 18 3月, 2017 2 次提交
  5. 17 3月, 2017 4 次提交
    • I
      Break stalls when no bg work is happening · d52f334c
      由 Islam AbdelRahman 提交于 
      Summary:
Current stall will keep sleeping even if there is no Flush/Compactions to wait for, I changed the logic to break the stall if we are not flushing or compacting

db_bench command used
```
# fillrandom
# memtable size = 10MB
# value size = 1 MB
# num = 1000
# use /dev/shm
./db_bench --benchmarks="fillrandom,stats" --value_size=1048576 --write_buffer_size=10485760 --num=1000 --delayed_write_rate=XXXXX  --db="/dev/shm/new_stall" | grep "Cumulative stall"
```

```
Current results

# delayed_write_rate = 1000 Kb/sec
Cumulative stall: 00:00:9.031 H:M:S

# delayed_write_rate = 200 Kb/sec
Cumulative stall: 00:00:22.314 H:M:S

# delayed_write_rate = 100 Kb/sec
Cumulative stall: 00:00:42.784 H:M:S

# delayed_write_rate = 50 Kb/sec
Cumulative stall: 00:01:23.785 H:M:S

# delayed_write_rate = 25 Kb/sec
Cumulative stall: 00:02:45.702 H:M:S
```

```
New results

# delayed_write_rate = 1000 Kb/sec
Cumulative stall: 00:00:9.017 H:M:S

# delayed_write_rate = 200 Kb/sec
Cumulative stall: 00
Closes https://github.com/facebook/rocksdb/pull/1884

Differential Revision: D4585439

Pulled By: IslamAbdelRahman

fbshipit-source-id: aed2198
      d52f334c
    • A
      fix db_bench rate limiter callsites · e66221ad
      由 Andrew Kryczka 提交于 
      Summary:
pass nullptr as stats object for db_bench-specific rate limiters since its stats are intended to capture background write activity only.
Closes https://github.com/facebook/rocksdb/pull/1997

Differential Revision: D4726806

Pulled By: ajkr

fbshipit-source-id: 8e4b225
      e66221ad
    • S
      Replace "DEPRECATED" comment to "not supported" · dbae438a
      由 Siying Dong 提交于 
      Summary:
"DEPRECATED" is ambiguous. Make it clear that those options not supported won't take effect.
Closes https://github.com/facebook/rocksdb/pull/1995

Differential Revision: D4724241

Pulled By: siying

fbshipit-source-id: 1e812b8
      dbae438a
    • I
      Support SstFileManager::SetDeleteRateBytesPerSecond() · 995618a8
      由 Islam AbdelRahman 提交于 
      Summary:
Update DeleteScheduler component to support changing delete rate in runtime by introducing
SstFileManager::SetDeleteRateBytesPerSecond()
Closes https://github.com/facebook/rocksdb/pull/1994

Differential Revision: D4719906

Pulled By: IslamAbdelRahman

fbshipit-source-id: e6b8d9e
      995618a8
  7. 16 3月, 2017 1 次提交
    • I
      Add macros to include file name and line number during Logging · e1916368
      由 Islam AbdelRahman 提交于 
      Summary:
current logging
```
2017/03/14-14:20:30.393432 7fedde9f5700 (Original Log Time 2017/03/14-14:20:30.393414) [default] Level summary: base level 1 max bytes base 268435456 files[1 0 0 0 0 0 0] max score 0.25
2017/03/14-14:20:30.393438 7fedde9f5700 [JOB 2] Try to delete WAL files size 61417909, prev total WAL file size 73820858, number of live WAL files 2.
2017/03/14-14:20:30.393464 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//MANIFEST-000001 type=3 #1 -- OK
2017/03/14-14:20:30.393472 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//000003.log type=0 #3 -- OK
2017/03/14-14:20:31.427103 7fedd49f1700 [default] New memtable created with log file: #9. Immutable memtables: 0.
2017/03/14-14:20:31.427179 7fedde9f5700 [JOB 3] Syncing log #6
2017/03/14-14:20:31.427190 7fedde9f5700 (Original Log Time 2017/03/14-14:20:31.427170) Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots allowed 1, compaction slots scheduled 1
2017/03/14-14:20:31.
Closes https://github.com/facebook/rocksdb/pull/1990

Differential Revision: D4708695

Pulled By: IslamAbdelRahman

fbshipit-source-id: cb8968f
      e1916368
  9. 15 3月, 2017 2 次提交
  11. 14 3月, 2017 5 次提交
    • M
      Pinnableslice (2nd attempt) · 11526252
      由 Maysam Yabandeh 提交于 
      Summary:
PinnableSlice

    Summary:
    Currently the point lookup values are copied to a string provided by the
    user. This incures an extra memcpy cost. This patch allows doing point lookup
    via a PinnableSlice which pins the source memory location (instead of
    copying their content) and releases them after the content is consumed
    by the user. The old API of Get(string) is translated to the new API
    underneath.

    Here is the summary for improvements:

    value 100 byte: 1.8% regular, 1.2% merge values
    value 1k byte: 11.5% regular, 7.5% merge values
    value 10k byte: 26% regular, 29.9% merge values
    The improvement for merge could be more if we extend this approach to
    pin the merge output and delay the full merge operation until the user
    actually needs it. We have put that for future work.

    PS:
    Sometimes we observe a small decrease in performance when switching from
    t5452014 to this patch but with the old Get(string) API. The d
Closes https://github.com/facebook/rocksdb/pull/1756

Differential Revision: D4391738

Pulled By: maysamyabandeh

fbshipit-source-id: 6f3edd3
      11526252
    • A
      update history.md for fixing the bug that skips keys · e5bd8def
      由 Aaron Gao 提交于 
      Summary: Closes https://github.com/facebook/rocksdb/pull/1986

Differential Revision: D4699152

Pulled By: siying

fbshipit-source-id: b18c32c
      e5bd8def
    • S
      Add a new SstFileWriter constructor without explicit comparator · 1ffbdfd9
      由 Sagar Vemuri 提交于 
      Summary:
The comparator param in SstFileWriter constructor is redundant as it already exists as a field in options. So the current SstFileWriter constructor should be deprecated in favor of a new one which does not take a comparator.
Note that the jni/java apis have not been touched yet.
Closes https://github.com/facebook/rocksdb/pull/1978

Differential Revision: D4685629

Pulled By: sagar0

fbshipit-source-id: 372ce96
      1ffbdfd9
    • R
      Add ability to search for key prefix in sst_dump tool · ebd5639b
      由 Reid Horuff 提交于 
      Summary:
Add the flag --prefix to the sst_dump tool
This flag is similar to, and exclusive from, the --from flag.

--prefix=0x00FF will return all rows prefixed with 0x00FF.
The --to flag may also be specified and will work as expected.

These changes were used to help in debugging the power cycle corruption issue and theses changes were tested by scanning through a udb.
Closes https://github.com/facebook/rocksdb/pull/1984

Differential Revision: D4691814

Pulled By: reidHoruff

fbshipit-source-id: 027f261
      ebd5639b
    • M
      Fix some bugs in MockEnv · e6725e8c
      由 Maysam Yabandeh 提交于 
      Summary:
Fixing some bugs in MockEnv so it be actually used.
Closes https://github.com/facebook/rocksdb/pull/1914

Differential Revision: D4609923

Pulled By: maysamyabandeh

fbshipit-source-id: ca25735
      e6725e8c
  13. 11 3月, 2017 1 次提交
  15. 10 3月, 2017 2 次提交
  17. 09 3月, 2017 6 次提交
  19. 08 3月, 2017 4 次提交
  21. 07 3月, 2017 3 次提交
    • M
      Expose DB::DeleteRange and WriteBath::DeleteRange in Java · 5f65dc87
      由 Matteo Merli 提交于 
      Summary:
Added JNI wrapper from `DeleteRange` methods
Closes https://github.com/facebook/rocksdb/pull/1951

Differential Revision: D4657746

Pulled By: yiwu-arbug

fbshipit-source-id: 3fc7ab8
      5f65dc87
    • R
      Set logs as getting flushed before releasing lock, race condition fix · 58b12dfe
      由 Reid Horuff 提交于 
      Summary:
Relating to #1903:

In MaybeFlushColumnFamilies() we want to modify the 'getting_flushed' flag before releasing the db mutex when SwitchMemtable() is called.

The following 2 actions need to be atomic in MaybeFlushColumnFamilies()
- getting_flushed is false on oldest log
- we determine that all CFs can be flushed to successfully release oldest log
- we set getting_flushed = true on the oldest log.
-------
- getting_flushed is false on oldest log
- we determine that all CFs can NOT be flushed to successfully release oldest log
- we set unable_to_flush_oldest_log_ = true on the oldest log.

#### In the 2pc case:

T1 enters function but is unable to flush all CFs to release log
T1 sets unable_to_flush_oldest_log_ = true
T1 begins flushing all CFs possible

T2 enters function but is unable to flush all CFs to release log
T2 sees unable_to_flush_oldes_log_ has been set so exits

T3 enters function and will be able to flush all CFs to release oldest log
T3 sets getting_flushed = true on oldes
Closes https://github.com/facebook/rocksdb/pull/1909

Differential Revision: D4646235

Pulled By: reidHoruff

fbshipit-source-id: c8d0447
      58b12dfe
    • S
      Move db_test and external_sst_file_test out of Travis's MAC OS run · f8a4ea02
      由 Siying Dong 提交于 
      Summary:
After we have db_basic_test and external_sst_file_basic_test, we don't need to run db_test and external_sst_file_test in Travis's MAC OS run anymore. Move it out.
Closes https://github.com/facebook/rocksdb/pull/1940

Differential Revision: D4659361

Pulled By: siying

fbshipit-source-id: e64e291
      f8a4ea02
  23. 06 3月, 2017 1 次提交
    • M
      Fix a bug in tests in options operator= · 534581a3
      由 Maysam Yabandeh 提交于 
      Summary:
Note: Using the default operator= is an unsafe approach for Options since it destructs shared_ptr in
the same order of their creation, in contrast to destructors which
destructs them in the opposite order of creation. One particular problme is
that the cache destructor might invoke callback functions that use Option
members such as statistics. To work around this problem, we manually call
destructor of table_facotry which eventually clears the block cache.
Closes https://github.com/facebook/rocksdb/pull/1950

Differential Revision: D4655473

Pulled By: maysamyabandeh

fbshipit-source-id: 6c4bbff
      534581a3
  25. 04 3月, 2017 3 次提交
  27. 03 3月, 2017 3 次提交