- 18 4月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: Seems pretty important. Closes https://github.com/facebook/rocksdb/pull/2161 Differential Revision: D4891097 Pulled By: yiwu-arbug fbshipit-source-id: 16dc0149dc89d426407fabab4a5ba1d6d1562269
-
- 17 4月, 2017 3 次提交
-
-
由 Jay Lee 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2164 Differential Revision: D4897389 Pulled By: yiwu-arbug fbshipit-source-id: fac15374ae7fef1ece70fd2b9018f2451f3c2f7c
-
由 Tamir Duberstein 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2159 Differential Revision: D4894483 Pulled By: yiwu-arbug fbshipit-source-id: 607e17e8ef5d30dce02c27be9d1de7d9f823b4ae
-
由 Tudor Bosman 提交于
Summary: Previously, the shared library (make shared_lib) was built with only one compile line, compiling all .cc files and linking the shared library in one step. That step would often take 10+ minutes on one machine, and could not take advantage of multiple CPUs (it's only one invocation of the compiler). This commit changes the shared_lib build to compile .o files individually (placing the resulting .o files in the directory shared-objects) and then link them into the shared library at the end, similarly to how the java static build (jls) does it. Tested by making sure that both static and shared libraries work, and by making sure that "make clean" cleans up the shared-objects directory. Closes https://github.com/facebook/rocksdb/pull/2165 Differential Revision: D4897121 Pulled By: yiwu-arbug fbshipit-source-id: 9811e043d1c01e10503593f3489d186c786ee7d7
-
- 15 4月, 2017 3 次提交
-
-
由 Aaron Gao 提交于
Summary: st_blocks shows 16 though the right value is 8. This happens occasionally which seems a bug. Closes https://github.com/facebook/rocksdb/pull/2160 Differential Revision: D4893542 Pulled By: lightmark fbshipit-source-id: 68e832586b58bbc6162efbe83ce273f1570d5be3
-
由 Aaron Gao 提交于
Summary: prefetch some data from the end of the file for each compaction to reduce IO. Closes https://github.com/facebook/rocksdb/pull/2149 Differential Revision: D4880576 Pulled By: lightmark fbshipit-source-id: aa767cd1afc84c541837fbf1ad6c0d45b34d3932
-
由 Siying Dong 提交于
Summary: Default to build using gcc-5. Only apply to Facebook-only environments. Closes https://github.com/facebook/rocksdb/pull/2158 Differential Revision: D4887568 Pulled By: siying fbshipit-source-id: 53496c9af3273ccd44441bd0bef9d29beefbc00b
-
- 14 4月, 2017 7 次提交
-
-
由 Yi Wu 提交于
Summary: The concept about early exit in write thread implementation is a confusing one. It means that if early exit is allowed, batch group leader will not responsible to exit the batch group, but the last finished writer do. In case we need to mark log synced, or encounter memtable insert error, early exit is disallowed. This patch remove such a concept by: * In all cases, the last finished writer (not necessary leader) is responsible to exit batch group. * In case of parallel memtable write, leader will also mark log synced after memtable insert and before signal finish (call `CompleteParallelWorker()`). The purpose is to allow mark log synced (which require locking mutex) can run in parallel to memtable insert in other writers. * The last finish writer should handle memtable insert error (update bg_error_) before exiting batch group. Closes https://github.com/facebook/rocksdb/pull/2134 Differential Revision: D4869667 Pulled By: yiwu-arbug fbshipit-source-id: aec170847c85b90f4179d6a4608a4fe1361544e3
-
由 Sagar Vemuri 提交于
Summary: When people are working off of a rocksdb fork, i.e. when their 'origin' points to github.com/<username>/rocksdb, the script creates a new branch and pushes to their origin. The new branch created by this script should instead be pushed to github.com/facebook/rocksdb. Many people might have named facebook/rocksdb remote as 'upstream' (or something else). This fix provides an option to specify the remote to push the branch to. The default is still 'origin' More context: When I created 5.4 branch using this script, it got pushed to sagar0/rocksdb instead of facebook/rocksdb, as I was working off of a fork. My 'origin' was pointing to sagar0/rocksdb. My 'upstream' was set to 'facebook/rocksdb'. So, I had to manually push the branch to my 'upstream'. Closes https://github.com/facebook/rocksdb/pull/2156 Differential Revision: D4885333 Pulled By: sagar0 fbshipit-source-id: 9410eab5bd9bbefc340059800bd6b8434406729d
-
由 Aaron Gao 提交于
Summary: Replace Options::use_direct_writes with Options::use_direct_io_for_flush_and_compaction Now if Options::use_direct_io_for_flush_and_compaction = true, we will enable direct io for both reads and writes for flush and compaction job. Whereas Options::use_direct_reads controls user reads like iterator and Get(). Closes https://github.com/facebook/rocksdb/pull/2117 Differential Revision: D4860912 Pulled By: lightmark fbshipit-source-id: d93575a8a5e780cf7e40797287edc425ee648c19
-
由 Aaron Gao 提交于
Summary: add the missing space Closes https://github.com/facebook/rocksdb/pull/2150 Differential Revision: D4880696 Pulled By: lightmark fbshipit-source-id: a4e0ad6a8ea45d6469d3f6c8514fdeb4cf10aaf5
-
由 Igor Canadi 提交于
Summary: BYTES_WRITTEN accounting doesn't work with disabled WAL. For example, this is what we get in the LOG: ``` Cumulative writes: 9794K writes, 228M keys, 9794K commit groups, 1.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s ``` WAL bytes are tracked in a different statistic: https://github.com/facebook/rocksdb/blob/master/db/internal_stats.h#L105. BYTES_WRITTEN should count all the writes. Closes https://github.com/facebook/rocksdb/pull/2133 Differential Revision: D4880615 Pulled By: yiwu-arbug fbshipit-source-id: 8fd0b223099f3f5ad7df79d4e737d313687fec69
-
由 Islam AbdelRahman 提交于
Summary: Update ShipIt to update push changes to TARGETS Reviewed By: sdwilsh Differential Revision: D4873590 fbshipit-source-id: bb5ab9bed7ad7bc51f1e2ee1fe8204224aaae2fb
-
由 Islam AbdelRahman 提交于
-
- 13 4月, 2017 3 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2142 Reviewed By: siying Differential Revision: D4874696 Pulled By: sagar0 fbshipit-source-id: 03e6e21735bb74e5a37cc913aabb2c250af558cc
-
由 Daniel Black 提交于
Summary: To correct a build process where the JAVA_TEST_LIBDIR is a symlink to a cache directory. Test -s (size 0) on symlinks returns true, resulting in a mkdir over the top of the symlink resulting in failure. As a solution -d checks if it is a directory (or the symlink refers to a directory), which works in the case of real directories and symlinks to directories. Trivial I know but it was really easy for me to use a symlink here to prevent frequent downloads in a CI environment. Thanks for your consideration. Closes https://github.com/facebook/rocksdb/pull/1917 Differential Revision: D4612263 Pulled By: siying fbshipit-source-id: 4d458f8e1760068cdd6b5eae4bce6e12c400df41
-
由 Siying Dong 提交于
Summary: Build Java and RocksDB LITE as a customized unit test under internal_repo_rocksdb. One thing I'm not sure is that whether these two tests are triggered in every flavor. Reviewed By: IslamAbdelRahman Differential Revision: D4855868 fbshipit-source-id: 82a1628b458744d7692bbd29ef7424cca1294031
-
- 12 4月, 2017 6 次提交
-
-
由 Islam AbdelRahman 提交于
-
由 Aaron Gao 提交于
Summary: Users usually set readahead buffer to a multiple of 4k, more than that, usually a multiple of blocks. So previously we set real buffer size 512 * n + 4k, which may introduce an additional block reading. Closes https://github.com/facebook/rocksdb/pull/2138 Differential Revision: D4871504 Pulled By: lightmark fbshipit-source-id: b070faa51d92e976e8e8468c00692699e585e243
-
由 Aaron Gao 提交于
Summary: filter the warning out and only print it once. Closes https://github.com/facebook/rocksdb/pull/2137 Differential Revision: D4870925 Pulled By: lightmark fbshipit-source-id: 91b363ce7f70bce88b0780337f408fc4649139b8
-
由 Siying Dong 提交于
Summary: In some CI test environment, compression libraries can't be successfully built. It still helps to build RocksDB there. Provide such an option to skip to download and build compression libraries. Closes https://github.com/facebook/rocksdb/pull/2135 Differential Revision: D4872617 Pulled By: siying fbshipit-source-id: bb21ac373bc62a2528cdf1ca4547e05fcae86214
-
由 Sagar Vemuri 提交于
Summary: Moved MergeOperatorPinning tests from db_test2.cc to db_merge_operator_test.cc. [This is the same code as PR #2104 , which has already been reviewed, but I am creating a new PR as I cannot import from #2104 onto phabricator anymore even after rebasing. I'll close and discard #2104.] Closes https://github.com/facebook/rocksdb/pull/2125 Differential Revision: D4863312 Pulled By: sagar0 fbshipit-source-id: 0f71a7690aa09c1d03ee85ce2bc1d2d89e4f4399
-
由 Maysam Yabandeh 提交于
Summary: This reverts commit 97ec8a13. Closes https://github.com/facebook/rocksdb/pull/2136 Differential Revision: D4870610 Pulled By: maysamyabandeh fbshipit-source-id: cdbfba135b065562f38f704f350a9a4e63a9a122
-
- 11 4月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: Currently level histogram is only printed out for DB stats and for default CF. This is confusing. Change to print for every CF instead. Closes https://github.com/facebook/rocksdb/pull/2126 Differential Revision: D4865373 Pulled By: siying fbshipit-source-id: 1c853e0ac66e00120ee931cabc9daf69ccc2d577
-
由 Manuel Ung 提交于
Summary: Upgrading a shared lock was silently succeeding because the actual locking code was skipped. This is because if the keys are tracked, it is assumed that they are already locked and do not require locking. Fix this by recording in tracked keys whether the key was locked exclusively or not. Note that lock downgrades are impossible, which is the behaviour we expect. This fixes facebook/mysql-5.6#587. Closes https://github.com/facebook/rocksdb/pull/2122 Differential Revision: D4861489 Pulled By: IslamAbdelRahman fbshipit-source-id: 58c7ebe7af098bf01b9774b666d3e9867747d8fd
-
由 Manuel Ung 提交于
Summary: Extend TransactionOptions to include max_write_batch_size which determines the maximum size of the writebatch representation. If memory limit is exceeded, the operation will abort with subcode kMemoryLimit. Closes https://github.com/facebook/rocksdb/pull/2124 Differential Revision: D4861842 Pulled By: lth fbshipit-source-id: 46fd172ea67cc90bbba829bf0d70cfab2261c161
-
由 Maysam Yabandeh 提交于
Summary: Run the time command before regression tests, parse the output, and add the numbers to the report. Closes https://github.com/facebook/rocksdb/pull/2101 Differential Revision: D4862781 Pulled By: maysamyabandeh fbshipit-source-id: 4a81caa5d14187d67093aad154c8f0ad56aba901
-
由 Maysam Yabandeh 提交于
Summary: also did minor refactoring Closes https://github.com/facebook/rocksdb/pull/2115 Differential Revision: D4855818 Pulled By: maysamyabandeh fbshipit-source-id: fbca6ac57e5c6677fffe8354f7291e596a50cb77
-
由 Sagar Vemuri 提交于
Summary: DBIter, and in-turn NewDBIterator and NewArenaWrappedDBIterator, take a bunch of params. They can be reduced by passing in ReadOptions directly instead of passing in every new param separately. It also seems much cleaner as a bunch of the params towards the end seem to be optional. (Recently I introduced max_skippable_internal_keys, which added one more to the already huge count). Idea courtesy IslamAbdelRahman Closes https://github.com/facebook/rocksdb/pull/2116 Differential Revision: D4857128 Pulled By: sagar0 fbshipit-source-id: 7d239df094b94bd9ea79d145cdf825478ac037a8
-
- 10 4月, 2017 1 次提交
-
-
由 Willem Jan Withagen 提交于
Summary: The compiler error: ``` /home/jenkins/workspace/ceph-master/src/rocksdb/db/db_impl.cc:20:10: fatal error: 'jemalloc/jemalloc.h' file not found ^ 1 error generated. ``` But is does compile with the `WITH_JEMALLOC` set. So ignore all the other settings. Closes https://github.com/facebook/rocksdb/pull/2118 Differential Revision: D4858387 Pulled By: yiwu-arbug fbshipit-source-id: 05b982969dcab53669a73a903641e71641c714e7
-
- 08 4月, 2017 7 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2114 Differential Revision: D4854860 Pulled By: IslamAbdelRahman fbshipit-source-id: b871c5b9ccc52d20f5ceacdd172dc70b1dbf9110
-
由 Islam AbdelRahman 提交于
Remove .arcconfig fbshipit-source-id: fe9f000138776a14c507b24c952ecf50b48d71a9
-
由 Islam AbdelRahman 提交于
fbshipit-source-id: 2e88ce2a16d16620aa6ef36aabbf3e91be2fc779
-
由 Maysam Yabandeh 提交于
Summary: Check the result of the benchmark againt a specified truth_db, which is expected to be produced using the same benchmark but perhaps on a different commit or with different configs. The verification is simple and assumes that key/values are generated deterministically. This assumption would break if db_bench using rand variable differently from the benchmark that produced truth_db. Currently it is checked to work on fillrandom and readwhilewriting. A param finish_after_writes is added to ensure that the background writing thread will write the same number of entries between two benchmarks. Example: $ TEST_TMPDIR=/dev/shm/truth_db ./db_bench --benchmarks="fillrandom,readwhilewriting" --num=200000 --finish_after_writes=true $ TEST_TMPDIR=/dev/shm/tmpdb ./db_bench --benchmarks="fillrandom,readwhilewriting,verify" --truth_db /dev/shm/truth_db/dbbench --num=200000 --finish_after_writes=true Verifying db <= truth_db... Verifying db >= truth_db... ...Verified Closes https://github.com/facebook/rocksdb/pull/2098 Differential Revision: D4839233 Pulled By: maysamyabandeh fbshipit-source-id: 2f4ed31
-
由 Ayappan 提交于
Summary: Fixes this issue https://github.com/facebook/rocksdb/issues/2108 Closes https://github.com/facebook/rocksdb/pull/2109 Differential Revision: D4851965 Pulled By: yiwu-arbug fbshipit-source-id: 6ee807b
-
由 Volker Mische 提交于
Summary: Currently the examples fail, having a job building them would prevent that from happening. Closes https://github.com/facebook/rocksdb/pull/2007 Differential Revision: D4831570 Pulled By: yiwu-arbug fbshipit-source-id: e38adfa
-
由 Nikhil Benesch 提交于
Summary: OpenBSD doesn't have `O_DIRECT`, so avoid it. (RocksDB compiles successfully on OpenBSD with this patch.) Closes https://github.com/facebook/rocksdb/pull/2106 Differential Revision: D4847833 Pulled By: siying fbshipit-source-id: 214b785
-
- 07 4月, 2017 3 次提交
-
-
由 Siying Dong 提交于
Summary: 1. Move universal compaction picker to separate files compaction_picker_universal.cc and compaction_picker_universal.h. 2. Rename some functions to make the code easier to understand. 3. Move leveled compaction picking code to a dedicated class, so that we we don't need to pass some common variable around when calling functions. It also allowed us to break down LevelCompactionPicker::PickCompaction() to smaller functions. Closes https://github.com/facebook/rocksdb/pull/2100 Differential Revision: D4845948 Pulled By: siying fbshipit-source-id: efa0ab4
-
由 Aaron Gao 提交于
Summary: In RocksDB, we sometimes preallocate the estimated space for a file to have better perf with fallocate (if supported). Usually it is a little bit bigger than the real resulting file size. At this time, we have to let the Filesystem reclaim the space not used. Ideally, calling ftruncate to truncate the file to its real size should be enough. HOWEVER, it isn't on tmpfs, which we witness in our case, with some buggy kernel version. ftruncate a file with preallocated space doesn't change number of the blocks used by the file, which means the space not used by the file is not returned to the filesystems. So in this case we need fallocate with FALLOC_FL_PUNCH_HOLE to explicitly reclaim the used blocks. It is a hack to cope with the kernel bug and usually we should not need it. Closes https://github.com/facebook/rocksdb/pull/2102 Differential Revision: D4848934 Pulled By: lightmark fbshipit-source-id: f1b40b5
-
由 Sagar Vemuri 提交于
Summary: This is an effort to club all string related utility functions into one common place, in string_util, so that it is easier for everyone to know what string processing functions are available. Right now they seem to be spread out across multiple modules, like logging and options_helper. Check the sub-commits for easier reviewing. Closes https://github.com/facebook/rocksdb/pull/2094 Differential Revision: D4837730 Pulled By: sagar0 fbshipit-source-id: 344278a
-