- 31 3月, 2017 2 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Operations like Seek/Next/Prev sometimes take too long to complete when there are many internal keys to be skipped. Adding an option, max_skippable_internal_keys -- which could be used to set a threshold for the maximum number of keys that can be skipped, will help to address these cases where it is much better to fail a request (as incomplete) than to wait for a considerable time for the request to complete. This feature -- to fail an iterator seek request as incomplete, is disabled by default when max_skippable_internal_keys = 0. It is enabled only when max_skippable_internal_keys > 0. This feature is based on the discussion mentioned in the PR https://github.com/facebook/rocksdb/pull/1084. Closes https://github.com/facebook/rocksdb/pull/2000 Differential Revision: D4753223 Pulled By: sagar0 fbshipit-source-id: 1c973f7
-
由 Herman Lee 提交于
Summary: instead of thread_local The cleanup path for the rocksdb database might not have the thread_updater_local_cache_ pointer initialized because the thread executing the cleanup is likely not a rocksdb thread. This results in a memory leak detected by Valgrind. The cleanup code path should use the thread_status_updater pointer obtained from the DB object instead of a thread local one. Closes https://github.com/facebook/rocksdb/pull/2059 Differential Revision: D4801611 Pulled By: hermanlee fbshipit-source-id: 407d7de
-
- 30 3月, 2017 5 次提交
-
-
由 Aaron Gao 提交于
Summary: add ldb to regression test in order to enable reuse db by creating checkpoint Closes https://github.com/facebook/rocksdb/pull/2030 Differential Revision: D4800549 Pulled By: lightmark fbshipit-source-id: d3a7325
-
由 Min Wei 提交于
Summary: Currently the fast crc32 path is not enabled on Windows. I am trying to enable it here, hopefully, with the minimum impact to the existing code structure. Closes https://github.com/facebook/rocksdb/pull/2033 Differential Revision: D4770635 Pulled By: siying fbshipit-source-id: 676f8b8
-
由 Mikhail Antonov 提交于
Summary: Adding API missing after https://github.com/facebook/rocksdb/commit/1ffbdfd9a7637b6517053842386d71df2cd00d9b#diff-b94146418eed4a9c1bf324041b95b279. adamretter IslamAbdelRahman Tested locally. Closes https://github.com/facebook/rocksdb/pull/2028 Differential Revision: D4762817 Pulled By: IslamAbdelRahman fbshipit-source-id: 833f478
-
由 Sharan Suryanarayanan 提交于
Summary: Changed the naming of backup engine threads from "ldb" to "backup_engine" Closes https://github.com/facebook/rocksdb/pull/2053 Differential Revision: D4799325 Pulled By: ajkr fbshipit-source-id: 046893f
-
由 Siying Dong 提交于
Summary: Add two DB properties: rocksdb.actual_delayed_write_rate and rocksdb.is_write_stooped, for people to know whether current writes are being throttled. Closes https://github.com/facebook/rocksdb/pull/2043 Differential Revision: D4782975 Pulled By: siying fbshipit-source-id: 6b2f5cf
-
- 29 3月, 2017 3 次提交
-
-
由 Aaron Gao 提交于
Summary: As discuss in this thread: https://www.facebook.com/groups/rocksdb.dev/permalink/1218043868294125/ We remove fallocate with FALLOC_FL_PUNCH_HOLE because the recent bug on xfs in kernel 4.x+ that align file size to page size even with FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE. Closes https://github.com/facebook/rocksdb/pull/2038 Differential Revision: D4779974 Pulled By: siying fbshipit-source-id: 5f54625
-
由 Yi Wu 提交于
Summary: ...to fix lite build error. Closes https://github.com/facebook/rocksdb/pull/2046 Differential Revision: D4785910 Pulled By: yiwu-arbug fbshipit-source-id: b591f27
-
由 Maysam Yabandeh 提交于
Summary: Allow the users to specify the target index partition size. With this patch an index partition is cut before its estimated in-memory size goes above the configured value for metadata_block_size. The filter partitions are still cut right after an index partition is cut. Closes https://github.com/facebook/rocksdb/pull/2041 Differential Revision: D4780216 Pulled By: maysamyabandeh fbshipit-source-id: 95a0831
-
- 28 3月, 2017 1 次提交
-
-
由 Ayappan 提交于
Summary: Right now, building rocksdbjava in PowerPC is broken due to JNI library name. I figured it out that "uname -m" and java's os.arch matches in PowerPC architecture. I made use of this advantage to fix the issue. More info can found from this issue --> https://github.com/facebook/rocksdb/issues/1317 Closes https://github.com/facebook/rocksdb/pull/2040 Differential Revision: D4779967 Pulled By: siying fbshipit-source-id: 259f939
-
- 25 3月, 2017 2 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Fixes #1961 which causes a segfault when filter_policy is nullptr and both pin_l0_filter_and_index_blocks_in_cache/cache_index_and_filter_blocks are set. Closes https://github.com/facebook/rocksdb/pull/2029 Differential Revision: D4764862 Pulled By: maysamyabandeh fbshipit-source-id: 05bd695
-
由 Shu Zhang 提交于
Summary: Added fifo benchmark to db_bench. One thing i am not sure is that i am using CompactRange() instead of CompactFiles(). (may cause performance skew because CompactionRange() is not happening in current thread?) For CompactFiles(), for some reason FIFO compaction doesn't work as expected. More insight is welcomed. I guess FIFO compaction doesn't work with file names? igorcanadi test cmd: ./db_bench --compaction_style=2 --benchmarks=fillseqdeterministic --disable_auto_compactions --num_levels=1 --fifo_compaction_max_table_files_size_mb=10 ---------------------- DB 0 LSM --------------------- Level[0]: /000014.sst(size: 4211014 bytes) fillseqdeterministic : 4.731 micros/op 211381 ops/sec; 23.4 MB/s Closes https://github.com/facebook/rocksdb/pull/1734 Differential Revision: D4774964 Pulled By: siying fbshipit-source-id: 9d08df6
-
- 24 3月, 2017 5 次提交
-
-
由 Andrew Kryczka 提交于
Summary: need to consistently include "rocksdb/persistent_cache.h" to fix internal build Closes https://github.com/facebook/rocksdb/pull/2034 Differential Revision: D4768101 Pulled By: ajkr fbshipit-source-id: 2ecb07f
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2032 Differential Revision: D4766260 Pulled By: siying fbshipit-source-id: 415daa4
-
由 Warren Falk 提交于
Summary: I've added functions to the C API to support WriteBatchWithIndex as requested in #1833. I've also added unit tests to c_test I've implemented the WriteBatchWithIndex variation of every function available for regular WriteBatch. And added additional functions unique to WriteBatchWithIndex. For now, the following is omitted: 1. The ability to create WriteBatchWithIndex's custom batch-only iterator as I'm not sure what its purpose is. It should be possible to add later if anyone wants it. 2. The ability to create the batch with a fallback comparator, since it appears to be unnecessary. I believe the column family comparator will be used for this, meaning those using a custom comparator can just use the column family variations. Closes https://github.com/facebook/rocksdb/pull/1985 Differential Revision: D4760039 Pulled By: siying fbshipit-source-id: 393227e
-
由 Siying Dong 提交于
Summary: Since non_shn CI was made to run in parallel, /dev/shm is automatically used. It defeated the purpose of the test to cover a non-ramfs file system. Closes https://github.com/facebook/rocksdb/pull/2031 Differential Revision: D4764804 Pulled By: siying fbshipit-source-id: 5666bda
-
由 Sagar Vemuri 提交于
Summary: options_file_example should be added in .gitignore so that it does not show up as an untracked file in `git status`. Closes https://github.com/facebook/rocksdb/pull/2026 Differential Revision: D4759402 Pulled By: sagar0 fbshipit-source-id: d7fe133
-
- 23 3月, 2017 9 次提交
-
-
由 Ayappan 提交于
Summary: PR to fix this issue -> https://github.com/facebook/rocksdb/issues/1926 Closes https://github.com/facebook/rocksdb/pull/1965 Differential Revision: D4682411 Pulled By: siying fbshipit-source-id: a519be1
-
由 Daniel Black 提交于
Summary: Errors where: db/version_set.cc:1535:20: error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture] [this](const Fsize& f1, const Fsize& f2) -> bool { ^ db/version_set.cc:1541:20: error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture] [this](const Fsize& f1, const Fsize& f2) -> bool { ^ db/db_test.cc:2983:27: error: lambda capture 'kNumPutsBeforeWaitForFlush' is not required to be captured for this use [-Werror,-Wunused-lambda-capture] auto gen_l0_kb = [this, kNumPutsBeforeWaitForFlush](int size) { ^ Closes https://github.com/facebook/rocksdb/pull/1972 Differential Revision: D4685991 Pulled By: siying fbshipit-source-id: 9125379
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2025 Differential Revision: D4759957 Pulled By: siying fbshipit-source-id: 5b5b625
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2024 Differential Revision: D4755420 Pulled By: siying fbshipit-source-id: 7332ab1
-
由 Aaron Gao 提交于
Summary: make total_log_size_ atomic to avoid overflow caused by data race. Closes https://github.com/facebook/rocksdb/pull/2019 Differential Revision: D4751391 Pulled By: siying fbshipit-source-id: fac01dd
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2021 Differential Revision: D4755324 Pulled By: siying fbshipit-source-id: c8d7955
-
由 Dmitri Smirnov 提交于
Summary: MemTableInserter default constructs Post processing info std::map. However, on Windows with 2015 STL the default constructed map still dynamically allocates one node which shows up on a profiler and we loose ~40% throughput on fillrandom benchmark. Solution: declare a map as std::aligned storage and optionally construct. This addresses https://github.com/facebook/rocksdb/issues/1976 Before: ------------------------------------------------------------------- Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillonly] fillrandom : 2.775 micros/op 360334 ops/sec; 280.4 MB/s Microseconds per write: Count: 10000000 Average: 2.7749 StdDev: 39.92 Min: 1 Median: 2.0826 Max: 26051 Percentiles: P50: 2.08 P75: 2.55 P99: 3.55 P99.9: 9.58 P99.99: 51.5**6 ------------------------------------------------------ After: Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillon Closes https://github.com/facebook/rocksdb/pull/2011 Differential Revision: D4740823 Pulled By: siying fbshipit-source-id: 1daaa2c
-
由 Siying Dong 提交于
Summary: PlainTable now supports non-mmap mode. We don't need to check it anymore. Closes https://github.com/facebook/rocksdb/pull/1882 Differential Revision: D4751643 Pulled By: siying fbshipit-source-id: ab14540
-
由 Maysam Yabandeh 提交于
Summary: This is the last split of this pull request: https://github.com/facebook/rocksdb/pull/1891 which includes the reader part as well as the tests. Closes https://github.com/facebook/rocksdb/pull/1961 Differential Revision: D4672216 Pulled By: maysamyabandeh fbshipit-source-id: 6a2b829
-
- 22 3月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: Add a parameter to Checkpoint::CreateCheckpoint() so that flush can be skipped if total log file size is within a threshold. Closes https://github.com/facebook/rocksdb/pull/1993 Differential Revision: D4719842 Pulled By: siying fbshipit-source-id: 4f9d9e1
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2018 Differential Revision: D4748448 Pulled By: siying fbshipit-source-id: a54c2f9
-
由 Leonidas Galanis 提交于
Summary: Removed max_grandparent_overlap_factor from benchmark.sh since it is not a valid option anymore. Closes https://github.com/facebook/rocksdb/pull/2015 Differential Revision: D4748229 Pulled By: lgalanis fbshipit-source-id: c3869ea
-
由 Aaron Gao 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2017 Differential Revision: D4747656 Pulled By: lightmark fbshipit-source-id: c52f160
-
由 Raza Hussain 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2016 Differential Revision: D4747003 Pulled By: yiwu-arbug fbshipit-source-id: c1e2c5a
-
由 Siying Dong 提交于
Summary: Previously, when DB write buffer size triggers, we always pick the CF with most data in its memtable to flush. This approach can minimize total flush happens. Change the behavior to always pick the oldest unflushed CF, which makes it the same behavior when max_total_wal_size hits. This approach will minimize size used by max_total_wal_size. Closes https://github.com/facebook/rocksdb/pull/1987 Differential Revision: D4703214 Pulled By: siying fbshipit-source-id: 9ff8b09
-
- 21 3月, 2017 3 次提交
-
-
由 Raza Hussain 提交于
Summary: Resolved the following issue: https://github.com/facebook/rocksdb/issues/1930 Closes https://github.com/facebook/rocksdb/pull/2004 Differential Revision: D4736764 Pulled By: yiwu-arbug fbshipit-source-id: 64fe0b7
-
由 Aaron Gao 提交于
Summary: add debug mode for better debugging refactor some regex change micros/op to ops/sec Closes https://github.com/facebook/rocksdb/pull/1999 Differential Revision: D4742806 Pulled By: lightmark fbshipit-source-id: 0fe3ae6
-
由 Maysam Yabandeh 提交于
Summary: Fixes #2008 Closes https://github.com/facebook/rocksdb/pull/2009 Differential Revision: D4739885 Pulled By: maysamyabandeh fbshipit-source-id: 36fba5d
-
- 18 3月, 2017 2 次提交
-
-
由 Tamir Duberstein 提交于
Summary: Fixes #1790. adamretter siying Closes https://github.com/facebook/rocksdb/pull/1996 Differential Revision: D4730449 Pulled By: IslamAbdelRahman fbshipit-source-id: 19944fa
-
由 Aaron Gao 提交于
Summary: in buffered io, the filesize_ is the real size. Closes https://github.com/facebook/rocksdb/pull/1991 Differential Revision: D4711433 Pulled By: lightmark fbshipit-source-id: ad604b9
-
- 17 3月, 2017 2 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Current stall will keep sleeping even if there is no Flush/Compactions to wait for, I changed the logic to break the stall if we are not flushing or compacting db_bench command used ``` # fillrandom # memtable size = 10MB # value size = 1 MB # num = 1000 # use /dev/shm ./db_bench --benchmarks="fillrandom,stats" --value_size=1048576 --write_buffer_size=10485760 --num=1000 --delayed_write_rate=XXXXX --db="/dev/shm/new_stall" | grep "Cumulative stall" ``` ``` Current results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.031 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00:00:22.314 H:M:S # delayed_write_rate = 100 Kb/sec Cumulative stall: 00:00:42.784 H:M:S # delayed_write_rate = 50 Kb/sec Cumulative stall: 00:01:23.785 H:M:S # delayed_write_rate = 25 Kb/sec Cumulative stall: 00:02:45.702 H:M:S ``` ``` New results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.017 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00 Closes https://github.com/facebook/rocksdb/pull/1884 Differential Revision: D4585439 Pulled By: IslamAbdelRahman fbshipit-source-id: aed2198
-
由 Andrew Kryczka 提交于
Summary: pass nullptr as stats object for db_bench-specific rate limiters since its stats are intended to capture background write activity only. Closes https://github.com/facebook/rocksdb/pull/1997 Differential Revision: D4726806 Pulled By: ajkr fbshipit-source-id: 8e4b225
-