- 23 3月, 2017 4 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2021 Differential Revision: D4755324 Pulled By: siying fbshipit-source-id: c8d7955
-
由 Dmitri Smirnov 提交于
Summary: MemTableInserter default constructs Post processing info std::map. However, on Windows with 2015 STL the default constructed map still dynamically allocates one node which shows up on a profiler and we loose ~40% throughput on fillrandom benchmark. Solution: declare a map as std::aligned storage and optionally construct. This addresses https://github.com/facebook/rocksdb/issues/1976 Before: ------------------------------------------------------------------- Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillonly] fillrandom : 2.775 micros/op 360334 ops/sec; 280.4 MB/s Microseconds per write: Count: 10000000 Average: 2.7749 StdDev: 39.92 Min: 1 Median: 2.0826 Max: 26051 Percentiles: P50: 2.08 P75: 2.55 P99: 3.55 P99.9: 9.58 P99.99: 51.5**6 ------------------------------------------------------ After: Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillon Closes https://github.com/facebook/rocksdb/pull/2011 Differential Revision: D4740823 Pulled By: siying fbshipit-source-id: 1daaa2c
-
由 Siying Dong 提交于
Summary: PlainTable now supports non-mmap mode. We don't need to check it anymore. Closes https://github.com/facebook/rocksdb/pull/1882 Differential Revision: D4751643 Pulled By: siying fbshipit-source-id: ab14540
-
由 Maysam Yabandeh 提交于
Summary: This is the last split of this pull request: https://github.com/facebook/rocksdb/pull/1891 which includes the reader part as well as the tests. Closes https://github.com/facebook/rocksdb/pull/1961 Differential Revision: D4672216 Pulled By: maysamyabandeh fbshipit-source-id: 6a2b829
-
- 22 3月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: Add a parameter to Checkpoint::CreateCheckpoint() so that flush can be skipped if total log file size is within a threshold. Closes https://github.com/facebook/rocksdb/pull/1993 Differential Revision: D4719842 Pulled By: siying fbshipit-source-id: 4f9d9e1
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2018 Differential Revision: D4748448 Pulled By: siying fbshipit-source-id: a54c2f9
-
由 Leonidas Galanis 提交于
Summary: Removed max_grandparent_overlap_factor from benchmark.sh since it is not a valid option anymore. Closes https://github.com/facebook/rocksdb/pull/2015 Differential Revision: D4748229 Pulled By: lgalanis fbshipit-source-id: c3869ea
-
由 Aaron Gao 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2017 Differential Revision: D4747656 Pulled By: lightmark fbshipit-source-id: c52f160
-
由 Raza Hussain 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2016 Differential Revision: D4747003 Pulled By: yiwu-arbug fbshipit-source-id: c1e2c5a
-
由 Siying Dong 提交于
Summary: Previously, when DB write buffer size triggers, we always pick the CF with most data in its memtable to flush. This approach can minimize total flush happens. Change the behavior to always pick the oldest unflushed CF, which makes it the same behavior when max_total_wal_size hits. This approach will minimize size used by max_total_wal_size. Closes https://github.com/facebook/rocksdb/pull/1987 Differential Revision: D4703214 Pulled By: siying fbshipit-source-id: 9ff8b09
-
- 21 3月, 2017 3 次提交
-
-
由 Raza Hussain 提交于
Summary: Resolved the following issue: https://github.com/facebook/rocksdb/issues/1930 Closes https://github.com/facebook/rocksdb/pull/2004 Differential Revision: D4736764 Pulled By: yiwu-arbug fbshipit-source-id: 64fe0b7
-
由 Aaron Gao 提交于
Summary: add debug mode for better debugging refactor some regex change micros/op to ops/sec Closes https://github.com/facebook/rocksdb/pull/1999 Differential Revision: D4742806 Pulled By: lightmark fbshipit-source-id: 0fe3ae6
-
由 Maysam Yabandeh 提交于
Summary: Fixes #2008 Closes https://github.com/facebook/rocksdb/pull/2009 Differential Revision: D4739885 Pulled By: maysamyabandeh fbshipit-source-id: 36fba5d
-
- 18 3月, 2017 2 次提交
-
-
由 Tamir Duberstein 提交于
Summary: Fixes #1790. adamretter siying Closes https://github.com/facebook/rocksdb/pull/1996 Differential Revision: D4730449 Pulled By: IslamAbdelRahman fbshipit-source-id: 19944fa
-
由 Aaron Gao 提交于
Summary: in buffered io, the filesize_ is the real size. Closes https://github.com/facebook/rocksdb/pull/1991 Differential Revision: D4711433 Pulled By: lightmark fbshipit-source-id: ad604b9
-
- 17 3月, 2017 4 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Current stall will keep sleeping even if there is no Flush/Compactions to wait for, I changed the logic to break the stall if we are not flushing or compacting db_bench command used ``` # fillrandom # memtable size = 10MB # value size = 1 MB # num = 1000 # use /dev/shm ./db_bench --benchmarks="fillrandom,stats" --value_size=1048576 --write_buffer_size=10485760 --num=1000 --delayed_write_rate=XXXXX --db="/dev/shm/new_stall" | grep "Cumulative stall" ``` ``` Current results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.031 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00:00:22.314 H:M:S # delayed_write_rate = 100 Kb/sec Cumulative stall: 00:00:42.784 H:M:S # delayed_write_rate = 50 Kb/sec Cumulative stall: 00:01:23.785 H:M:S # delayed_write_rate = 25 Kb/sec Cumulative stall: 00:02:45.702 H:M:S ``` ``` New results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.017 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00 Closes https://github.com/facebook/rocksdb/pull/1884 Differential Revision: D4585439 Pulled By: IslamAbdelRahman fbshipit-source-id: aed2198
-
由 Andrew Kryczka 提交于
Summary: pass nullptr as stats object for db_bench-specific rate limiters since its stats are intended to capture background write activity only. Closes https://github.com/facebook/rocksdb/pull/1997 Differential Revision: D4726806 Pulled By: ajkr fbshipit-source-id: 8e4b225
-
由 Siying Dong 提交于
Summary: "DEPRECATED" is ambiguous. Make it clear that those options not supported won't take effect. Closes https://github.com/facebook/rocksdb/pull/1995 Differential Revision: D4724241 Pulled By: siying fbshipit-source-id: 1e812b8
-
由 Islam AbdelRahman 提交于
Summary: Update DeleteScheduler component to support changing delete rate in runtime by introducing SstFileManager::SetDeleteRateBytesPerSecond() Closes https://github.com/facebook/rocksdb/pull/1994 Differential Revision: D4719906 Pulled By: IslamAbdelRahman fbshipit-source-id: e6b8d9e
-
- 16 3月, 2017 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: current logging ``` 2017/03/14-14:20:30.393432 7fedde9f5700 (Original Log Time 2017/03/14-14:20:30.393414) [default] Level summary: base level 1 max bytes base 268435456 files[1 0 0 0 0 0 0] max score 0.25 2017/03/14-14:20:30.393438 7fedde9f5700 [JOB 2] Try to delete WAL files size 61417909, prev total WAL file size 73820858, number of live WAL files 2. 2017/03/14-14:20:30.393464 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//MANIFEST-000001 type=3 #1 -- OK 2017/03/14-14:20:30.393472 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//000003.log type=0 #3 -- OK 2017/03/14-14:20:31.427103 7fedd49f1700 [default] New memtable created with log file: #9. Immutable memtables: 0. 2017/03/14-14:20:31.427179 7fedde9f5700 [JOB 3] Syncing log #6 2017/03/14-14:20:31.427190 7fedde9f5700 (Original Log Time 2017/03/14-14:20:31.427170) Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots allowed 1, compaction slots scheduled 1 2017/03/14-14:20:31. Closes https://github.com/facebook/rocksdb/pull/1990 Differential Revision: D4708695 Pulled By: IslamAbdelRahman fbshipit-source-id: cb8968f
-
- 15 3月, 2017 2 次提交
-
-
由 Aaron Gao 提交于
Summary: remove unnecessary field `direct_io_`, use `use_direct_io()` instead. Closes https://github.com/facebook/rocksdb/pull/1992 Differential Revision: D4712195 Pulled By: lightmark fbshipit-source-id: 57d34f9
-
由 slyrz 提交于
Summary: This PR copies the missing CompressionType values ```cpp enum CompressionType : unsigned char { ... kXpressCompression = 0x6, kZSTD = 0x7, ... }; ``` into the C header. Closes https://github.com/facebook/rocksdb/pull/1989 Differential Revision: D4706265 Pulled By: siying fbshipit-source-id: e65e62a
-
- 14 3月, 2017 5 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: PinnableSlice Summary: Currently the point lookup values are copied to a string provided by the user. This incures an extra memcpy cost. This patch allows doing point lookup via a PinnableSlice which pins the source memory location (instead of copying their content) and releases them after the content is consumed by the user. The old API of Get(string) is translated to the new API underneath. Here is the summary for improvements: value 100 byte: 1.8% regular, 1.2% merge values value 1k byte: 11.5% regular, 7.5% merge values value 10k byte: 26% regular, 29.9% merge values The improvement for merge could be more if we extend this approach to pin the merge output and delay the full merge operation until the user actually needs it. We have put that for future work. PS: Sometimes we observe a small decrease in performance when switching from t5452014 to this patch but with the old Get(string) API. The d Closes https://github.com/facebook/rocksdb/pull/1756 Differential Revision: D4391738 Pulled By: maysamyabandeh fbshipit-source-id: 6f3edd3
-
由 Aaron Gao 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/1986 Differential Revision: D4699152 Pulled By: siying fbshipit-source-id: b18c32c
-
由 Sagar Vemuri 提交于
Summary: The comparator param in SstFileWriter constructor is redundant as it already exists as a field in options. So the current SstFileWriter constructor should be deprecated in favor of a new one which does not take a comparator. Note that the jni/java apis have not been touched yet. Closes https://github.com/facebook/rocksdb/pull/1978 Differential Revision: D4685629 Pulled By: sagar0 fbshipit-source-id: 372ce96
-
由 Reid Horuff 提交于
Summary: Add the flag --prefix to the sst_dump tool This flag is similar to, and exclusive from, the --from flag. --prefix=0x00FF will return all rows prefixed with 0x00FF. The --to flag may also be specified and will work as expected. These changes were used to help in debugging the power cycle corruption issue and theses changes were tested by scanning through a udb. Closes https://github.com/facebook/rocksdb/pull/1984 Differential Revision: D4691814 Pulled By: reidHoruff fbshipit-source-id: 027f261
-
由 Maysam Yabandeh 提交于
Summary: Fixing some bugs in MockEnv so it be actually used. Closes https://github.com/facebook/rocksdb/pull/1914 Differential Revision: D4609923 Pulled By: maysamyabandeh fbshipit-source-id: ca25735
-
- 11 3月, 2017 1 次提交
-
-
由 Min Wei 提交于
Summary: Without the cast, the build will break on Windows. Closes https://github.com/facebook/rocksdb/pull/1982 Differential Revision: D4690462 Pulled By: ajkr fbshipit-source-id: c493b6c
-
- 10 3月, 2017 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/1971 Differential Revision: D4683851 Pulled By: ajkr fbshipit-source-id: 967116e
-
由 Maysam Yabandeh 提交于
Summary: This reverts commit d43adf21. The patch has caused problems in regression tests. Will revert it for now until we figure how to debug the problems regression tests. Closes https://github.com/facebook/rocksdb/pull/1975 Differential Revision: D4682880 Pulled By: maysamyabandeh fbshipit-source-id: 84df83a
-
- 09 3月, 2017 6 次提交
-
-
由 Andrew Kryczka 提交于
Summary: lately it fails on travis due to a compiler bug (see https://github.com/google/googletest/issues/322#issuecomment-125645145). interestingly it seems to affect occurrences of `ASSERT_EQ(false, ...);` but not `ASSERT_EQ(true, ...);`. Closes https://github.com/facebook/rocksdb/pull/1958 Differential Revision: D4680742 Pulled By: ajkr fbshipit-source-id: 291fe41
-
由 Andrew Kryczka 提交于
Summary: Domas enlightened me about p100 (i.e., max) stats. Let's add them to our histograms. Closes https://github.com/facebook/rocksdb/pull/1968 Differential Revision: D4678716 Pulled By: ajkr fbshipit-source-id: 65e7118
-
由 Maysam Yabandeh 提交于
Summary: It augments the regression benchmarks with a time command, parses the output, and print them to the SUMMARY.csv file. I tested a variation of the script locally. Any idea how to do run a test that also involves writing to scuba tables? Closes https://github.com/facebook/rocksdb/pull/1967 Differential Revision: D4679470 Pulled By: maysamyabandeh fbshipit-source-id: 44dac30
-
由 Andrew Kryczka 提交于
Summary: the 50%+ drained constraint wasn't working consistently in some of our test environments, maybe their resources are too low. relax the constraints a bit. Closes https://github.com/facebook/rocksdb/pull/1970 Differential Revision: D4679419 Pulled By: ajkr fbshipit-source-id: 3789cd8
-
由 Aaron Gao 提交于
Summary: fix db_iter bug introduced by [facebook#1413](https://github.com/facebook/rocksdb/pull/1413) Closes https://github.com/facebook/rocksdb/pull/1962 Differential Revision: D4672369 Pulled By: lightmark fbshipit-source-id: 6a22953
-
由 Dmitri Smirnov 提交于
Summary: This fixes an issue when the most recent readers assume that alignment is always set even if direct io is off. Also adjust slightly appveyor script to run db_basic_test cases concurrently. Closes https://github.com/facebook/rocksdb/pull/1959 Differential Revision: D4671972 Pulled By: IslamAbdelRahman fbshipit-source-id: 1886620
-
- 08 3月, 2017 4 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: new blog post for bulkoading Closes https://github.com/facebook/rocksdb/pull/1883 Differential Revision: D4671984 Pulled By: IslamAbdelRahman fbshipit-source-id: 3450860
-
由 Maysam Yabandeh 提交于
Summary: This is the second split of this pull request: https://github.com/facebook/rocksdb/pull/1891 which includes only the builder part. The testing will be included in the third split, where the reader is also included. Closes https://github.com/facebook/rocksdb/pull/1952 Differential Revision: D4660272 Pulled By: maysamyabandeh fbshipit-source-id: 36b3cf0
-
由 Sagar Vemuri 提交于
Summary: This PR is to support a way to iterate over all the keys that are just in memtables. Closes https://github.com/facebook/rocksdb/pull/1953 Differential Revision: D4663500 Pulled By: sagar0 fbshipit-source-id: 144e177
-
由 Leonidas Galanis 提交于
Summary: db_sst_test had been flaky occasionally in the following way: reached_max_space_on_compaction can in very rare cases be 0. This happens when the limit on maximum allowable space set using SetMaxAllowedSpaceUsage is hit during flush for all test db sizes (1,2,4,8 and 10MB).The fix clears the error returned when the the space limit is reached during flush. This ensures that the compaction call back will always be called. The runtime is increased slightly because the 1MB loop writes more data and hits the limit during multiple flushes until compaction is scheduled. Closes https://github.com/facebook/rocksdb/pull/1861 Differential Revision: D4557396 Pulled By: lgalanis fbshipit-source-id: ff778d1
-