- 08 4月, 2017 7 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2114 Differential Revision: D4854860 Pulled By: IslamAbdelRahman fbshipit-source-id: b871c5b9ccc52d20f5ceacdd172dc70b1dbf9110
-
由 Islam AbdelRahman 提交于
Remove .arcconfig fbshipit-source-id: fe9f000138776a14c507b24c952ecf50b48d71a9
-
由 Islam AbdelRahman 提交于
fbshipit-source-id: 2e88ce2a16d16620aa6ef36aabbf3e91be2fc779
-
由 Maysam Yabandeh 提交于
Summary: Check the result of the benchmark againt a specified truth_db, which is expected to be produced using the same benchmark but perhaps on a different commit or with different configs. The verification is simple and assumes that key/values are generated deterministically. This assumption would break if db_bench using rand variable differently from the benchmark that produced truth_db. Currently it is checked to work on fillrandom and readwhilewriting. A param finish_after_writes is added to ensure that the background writing thread will write the same number of entries between two benchmarks. Example: $ TEST_TMPDIR=/dev/shm/truth_db ./db_bench --benchmarks="fillrandom,readwhilewriting" --num=200000 --finish_after_writes=true $ TEST_TMPDIR=/dev/shm/tmpdb ./db_bench --benchmarks="fillrandom,readwhilewriting,verify" --truth_db /dev/shm/truth_db/dbbench --num=200000 --finish_after_writes=true Verifying db <= truth_db... Verifying db >= truth_db... ...Verified Closes https://github.com/facebook/rocksdb/pull/2098 Differential Revision: D4839233 Pulled By: maysamyabandeh fbshipit-source-id: 2f4ed31
-
由 Ayappan 提交于
Summary: Fixes this issue https://github.com/facebook/rocksdb/issues/2108 Closes https://github.com/facebook/rocksdb/pull/2109 Differential Revision: D4851965 Pulled By: yiwu-arbug fbshipit-source-id: 6ee807b
-
由 Volker Mische 提交于
Summary: Currently the examples fail, having a job building them would prevent that from happening. Closes https://github.com/facebook/rocksdb/pull/2007 Differential Revision: D4831570 Pulled By: yiwu-arbug fbshipit-source-id: e38adfa
-
由 Nikhil Benesch 提交于
Summary: OpenBSD doesn't have `O_DIRECT`, so avoid it. (RocksDB compiles successfully on OpenBSD with this patch.) Closes https://github.com/facebook/rocksdb/pull/2106 Differential Revision: D4847833 Pulled By: siying fbshipit-source-id: 214b785
-
- 07 4月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: 1. Move universal compaction picker to separate files compaction_picker_universal.cc and compaction_picker_universal.h. 2. Rename some functions to make the code easier to understand. 3. Move leveled compaction picking code to a dedicated class, so that we we don't need to pass some common variable around when calling functions. It also allowed us to break down LevelCompactionPicker::PickCompaction() to smaller functions. Closes https://github.com/facebook/rocksdb/pull/2100 Differential Revision: D4845948 Pulled By: siying fbshipit-source-id: efa0ab4
-
由 Aaron Gao 提交于
Summary: In RocksDB, we sometimes preallocate the estimated space for a file to have better perf with fallocate (if supported). Usually it is a little bit bigger than the real resulting file size. At this time, we have to let the Filesystem reclaim the space not used. Ideally, calling ftruncate to truncate the file to its real size should be enough. HOWEVER, it isn't on tmpfs, which we witness in our case, with some buggy kernel version. ftruncate a file with preallocated space doesn't change number of the blocks used by the file, which means the space not used by the file is not returned to the filesystems. So in this case we need fallocate with FALLOC_FL_PUNCH_HOLE to explicitly reclaim the used blocks. It is a hack to cope with the kernel bug and usually we should not need it. Closes https://github.com/facebook/rocksdb/pull/2102 Differential Revision: D4848934 Pulled By: lightmark fbshipit-source-id: f1b40b5
-
由 Sagar Vemuri 提交于
Summary: This is an effort to club all string related utility functions into one common place, in string_util, so that it is easier for everyone to know what string processing functions are available. Right now they seem to be spread out across multiple modules, like logging and options_helper. Check the sub-commits for easier reviewing. Closes https://github.com/facebook/rocksdb/pull/2094 Differential Revision: D4837730 Pulled By: sagar0 fbshipit-source-id: 344278a
-
由 Ayappan 提交于
Summary: While running `make jtest` using IBM Java, it fails at compactRangeToLevel with the below error. ``` Run: org.rocksdb.RocksDBTest testing now -> compactRangeToLevel JVMJNCK056E JNI error in ReleaseByteArrayElements: Got memory 0x00003FFF94AA8908 from object 0x00000000000C7F78, releasing from 0x00000000000C7F68 JVMJNCK077E Error detected in org/rocksdb/RocksDB.compactRange0(J[BI[BIZII)V JVMJNCK024E JNI error detected. Aborting. JVMJNCK025I Use -Xcheck:jni:nonfatal to continue running when errors are detected. Fatal error: JNI error Makefile:205: recipe for target 'run_test' failed make[1]: *** [run_test] Error 87 make[1]: Leaving directory '/home/ubuntu/rocksdb/java' Makefile:1542: recipe for target 'jtest' failed make: *** [jtest] Error 2 ``` After checking the code, it is vivid that we are messing up the `ReleaseByteArrayElements` args in `rocksdb_compactrange_helper`. ``` ................. 1959 s = db->CompactRange(compact_options, &begin_slice, &end_slice); 1960 } Closes https://github.com/facebook/rocksdb/pull/2060 Differential Revision: D4831427 Pulled By: yiwu-arbug fbshipit-source-id: dd02037
-
由 Yi Wu 提交于
Summary: Move memtable related files into memtable directory. Closes https://github.com/facebook/rocksdb/pull/2087 Differential Revision: D4829242 Pulled By: yiwu-arbug fbshipit-source-id: ca70ab6
-
由 Tamir Duberstein 提交于
Summary: siying this is a resubmission of #2081 with the 4th commit fixed. From that commit message: > Note that the previous use of quotes in PLATFORM_{CC,CXX}FLAGS was incorrect and caused GCC to produce the incorrect define: > > #define ROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE 1 > > This was the cause of the Linux build failure on the previous version of this change. I've tested this locally, and the Linux build succeeds now. Closes https://github.com/facebook/rocksdb/pull/2097 Differential Revision: D4839964 Pulled By: siying fbshipit-source-id: cc51322
-
- 06 4月, 2017 7 次提交
-
-
由 Siying Dong 提交于
Summary: Move some files under util/ to new directories env/, monitoring/ options/ and cache/ Closes https://github.com/facebook/rocksdb/pull/2090 Differential Revision: D4833681 Pulled By: siying fbshipit-source-id: 2fd8bef
-
由 Islam AbdelRahman 提交于
Summary: Current ``` ** Compaction Stats [default] ** Level Files Size(MB} Score Read(GB} Rn(GB} Rnp1(GB} Write(GB} Wnew(GB} Moved(GB} W-Amp Rd(MB/s} Wr(MB/s} Comp(sec} Comp(cnt} Avg(sec} KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 2/0 49.02 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.1 1 2 0.322 0 0 Sum 2/0 49.02 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 1 2 0.322 0 0 Int 0/0 0.00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 1 2 0.322 0 0 ``` New ``` ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn Key Closes https://github.com/facebook/rocksdb/pull/2055 Differential Revision: D4804576 Pulled By: IslamAbdelRahman fbshipit-source-id: 719be6a
-
由 Siying Dong 提交于
Summary: db_impl.cc is too large to manage. Divide db_impl.cc into db/db_impl.cc, db/db_impl_compaction_flush.cc, db/db_impl_files.cc, db/db_impl_open.cc and db/db_impl_write.cc. Closes https://github.com/facebook/rocksdb/pull/2095 Differential Revision: D4838188 Pulled By: siying fbshipit-source-id: c5f3059
-
由 Aaron Gao 提交于
Summary: This reverts commit 0fd57492. It breaks tmpfs on kernel 4.0 or earlier. We will wait for the fix before remove this part Closes https://github.com/facebook/rocksdb/pull/2096 Differential Revision: D4839661 Pulled By: lightmark fbshipit-source-id: 574a51f
-
由 Maysam Yabandeh 提交于
Summary: I tagged it experimental since the configuring the filter partitions by size will not be ready for this release. Closes https://github.com/facebook/rocksdb/pull/2049 Differential Revision: D4831575 Pulled By: maysamyabandeh fbshipit-source-id: fcc2d25
-
由 Aaron Gao 提交于
Summary: Rebuilding regression db at the first run every monday. Closes https://github.com/facebook/rocksdb/pull/2093 Differential Revision: D4836961 Pulled By: lightmark fbshipit-source-id: 22d6c25
-
由 Dmitri Smirnov 提交于
Summary: Rework test running script. New options SuiteRun - runs specified executables as google suite test cases in parallel. Run - this option now runs executables in parallel the same as 'tests' RunAll - scans for test executables and attempts to run them all as suites except those mentiones in RunOnly (hardcoded in the script) or specified either in $ExcludeTestCases $ExcludeTestExe Closes https://github.com/facebook/rocksdb/pull/2089 Differential Revision: D4832212 Pulled By: yiwu-arbug fbshipit-source-id: 954990c
-
- 05 4月, 2017 8 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Level-based L0->L0 compaction operates on spans of files that aren't currently being compacted. It reduces the number of L0 files, thus making write stall conditions harder to reach. - L0->L0 is triggered when base level is unavailable due to pending compactions - L0->L0 always outputs one file of at most `max_level0_burst_file_size` bytes. - Subcompactions are disabled for L0->L0 since we want to output one file. - Input files are chosen as the longest span of available files that will fit within the size limit. This minimizes number of files in L0. Closes https://github.com/facebook/rocksdb/pull/2027 Differential Revision: D4760318 Pulled By: ajkr fbshipit-source-id: 9d07183
-
由 Jevon Qiao 提交于
Summary: Add a notice on gflags installation to help people build and install RocksDB from source code correctly. Please see https://github.com/facebook/rocksdb/issues/1775 Closes https://github.com/facebook/rocksdb/pull/2061 Differential Revision: D4831323 Pulled By: yiwu-arbug fbshipit-source-id: 0df1f0e
-
由 Siying Dong 提交于
fbshipit-source-id: 43b4529
-
由 Islam AbdelRahman 提交于
Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
-
由 Tamir Duberstein 提交于
Summary: See individual commits. yuslepukhin siying Closes https://github.com/facebook/rocksdb/pull/2081 Differential Revision: D4824639 Pulled By: IslamAbdelRahman fbshipit-source-id: 2fc2b00
-
由 Aaron Gao 提交于
Summary: to void future bug that caused by the mix of userkey/internalkey Closes https://github.com/facebook/rocksdb/pull/2084 Differential Revision: D4825889 Pulled By: lightmark fbshipit-source-id: 28411db
-
由 Andrew Kryczka 提交于
Summary: I've needed Env timing measurements a few times now, so finally built something for it. Closes https://github.com/facebook/rocksdb/pull/2073 Differential Revision: D4811231 Pulled By: ajkr fbshipit-source-id: 218a249
-
由 Yi Wu 提交于
Summary: Refactor WriteImpl() so when I plug-in the pipeline write code (which is an alternative approach for WriteThread), some of the logic can be reuse. I split out the following methods from WriteImpl(): * PreprocessWrite() * HandleWALFull() (previous MaybeFlushColumnFamilies()) * HandleWriteBufferFull() * WriteToWAL() Also adding a constructor to WriteThread::Writer, and move WriteContext into db_impl.h. No real logic change in this patch. Closes https://github.com/facebook/rocksdb/pull/2042 Differential Revision: D4781014 Pulled By: yiwu-arbug fbshipit-source-id: d45ca18
-
- 04 4月, 2017 8 次提交
-
-
由 Siying Dong 提交于
Summary: It is confusing to have auto_roll_logger to stay under db/, which has nothing to do with database. Move filename together as it is a dependency. Closes https://github.com/facebook/rocksdb/pull/2080 Differential Revision: D4821141 Pulled By: siying fbshipit-source-id: ca7d768
-
由 Maysam Yabandeh 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2037 Differential Revision: D4822085 Pulled By: maysamyabandeh fbshipit-source-id: 9d5a986
-
由 Aaron Gao 提交于
Summary: add \' pair to avoid bash expanding before run remotely Closes https://github.com/facebook/rocksdb/pull/2079 Differential Revision: D4821342 Pulled By: lightmark fbshipit-source-id: 418ba41
-
由 Doogie Lee 提交于
Summary: Following the instructions on INSTALL.md, I found some errors while installing gflags and zstandard libraries on Centos and fixed them. Thanks :) Closes https://github.com/facebook/rocksdb/pull/2077 Differential Revision: D4820804 Pulled By: ajkr fbshipit-source-id: db4abb3
-
由 Siying Dong 提交于
Summary: auto_roll_logger_test relies on timing conditon that some operations finish within 1 seconds. This caused flaky tests. Move away from real timing and sleep and use fake time to verify the time-based rolling. Closes https://github.com/facebook/rocksdb/pull/2066 Differential Revision: D4810647 Pulled By: siying fbshipit-source-id: c54d994
-
由 Nikhil Benesch 提交于
Summary: `uint` is nonstandard and not a built-in type on all compilers; replace it with the always-valid `unsigned int`. I assume this went unnoticed because it's inside an `#ifdef ROCKDB_JEMALLOC`. Closes https://github.com/facebook/rocksdb/pull/2075 Differential Revision: D4820427 Pulled By: ajkr fbshipit-source-id: 0876561
-
由 Andrew Kryczka 提交于
Summary: Previously it always showed 0.0 for L0 write-amp because we were dividing by bytes read from non-output level. For L0, we should instead divide by bytes ingested to the DB. Note the numerator (bytes written to L0) includes flush bytes. Closes https://github.com/facebook/rocksdb/pull/2078 Differential Revision: D4816902 Pulled By: ajkr fbshipit-source-id: 7dca31a
-
由 Tamir Duberstein 提交于
Summary: yuslepukhin See individual commits. Closes https://github.com/facebook/rocksdb/pull/2051 Differential Revision: D4819697 Pulled By: siying fbshipit-source-id: 75c1de4
-
- 01 4月, 2017 4 次提交
-
-
由 Aaron Gao 提交于
Summary: avoid exit when $? != 0 Closes https://github.com/facebook/rocksdb/pull/2076 Differential Revision: D4815655 Pulled By: lightmark fbshipit-source-id: 829299e
-
由 Aaron Gao 提交于
Summary: Fix the bug that previous test existence locally Closes https://github.com/facebook/rocksdb/pull/2074 Differential Revision: D4813631 Pulled By: lightmark fbshipit-source-id: eceb0d9
-
由 Aaron Gao 提交于
Summary: force remove db dir when rebuilding Closes https://github.com/facebook/rocksdb/pull/2067 Differential Revision: D4811926 Pulled By: lightmark fbshipit-source-id: ab068a2
-
由 Adam Retter 提交于
Summary: Closes https://github.com/facebook/rocksdb/issues/2068 Closes https://github.com/facebook/rocksdb/pull/2072 Differential Revision: D4810855 Pulled By: ajkr fbshipit-source-id: 288ddb9
-