- 18 7月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Unfortunately we can't use -DNDEBUG yet since we don't properly exclude the test libraries/executables from the non-debug builds on non-MSVC platforms. Previously this was failing on Linux for every build type except `CMAKE_BUILD_TYPE=Debug`. Reverts a48a62d5 Closes https://github.com/facebook/rocksdb/pull/2595 Differential Revision: D5436182 Pulled By: ajkr fbshipit-source-id: 062f07cc9ce06a073b66054722b27bac1890dca3
-
- 11 7月, 2017 1 次提交
-
-
由 Giuseppe Ottaviano 提交于
Summary: Instead of ignoring UBSan checks, fix the negative shifts in Hash(). Also add test to make sure the hash values are stable over time. The values were computed before this change, so the test also verifies the correctness of the change. Closes https://github.com/facebook/rocksdb/pull/2546 Differential Revision: D5386369 Pulled By: yiwu-arbug fbshipit-source-id: 6de4b44461a544d6222cc5d72d8cda2c0373d17e
-
- 28 6月, 2017 1 次提交
-
-
由 Yi Wu 提交于
Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
-
- 27 6月, 2017 1 次提交
-
-
由 Ewout Prangsma 提交于
Summary: This PR adds support for encrypting data stored by RocksDB when written to disk. It adds an `EncryptedEnv` override of the `Env` class with matching overrides for sequential&random access files. The encryption itself is done through a configurable `EncryptionProvider`. This class creates is asked to create `BlockAccessCipherStream` for a file. This is where the actual encryption/decryption is being done. Currently there is a Counter mode implementation of `BlockAccessCipherStream` with a `ROT13` block cipher (NOTE the `ROT13` is for demo purposes only!!). The Counter operation mode uses an initial counter & random initialization vector (IV). Both are created randomly for each file and stored in a 4K (default size) block that is prefixed to that file. The `EncryptedEnv` implementation is such that clients of the `Env` class do not see this prefix (nor data, nor in filesize). The largest part of the prefix block is also encrypted, and there is room left for implementation specific settings/values/keys in there. To test the encryption, the `DBTestBase` class has been extended to consider a new environment variable called `ENCRYPTED_ENV`. If set, the test will setup a encrypted instance of the `Env` class to use for all tests. Typically you would run it like this: ``` ENCRYPTED_ENV=1 make check_some ``` There is also an added test that checks that some data inserted into the database is or is not "visible" on disk. With `ENCRYPTED_ENV` active it must not find plain text strings, with `ENCRYPTED_ENV` unset, it must find the plain text strings. Closes https://github.com/facebook/rocksdb/pull/2424 Differential Revision: D5322178 Pulled By: sdwilsh fbshipit-source-id: 253b0a9c2c498cc98f580df7f2623cbf7678a27f
-
- 17 6月, 2017 1 次提交
-
-
由 Chen Shen 提交于
Summary: This PR implements the MergeOperator for Cassandra Row Values. Closes https://github.com/facebook/rocksdb/pull/2289 Differential Revision: D5055464 Pulled By: scv119 fbshipit-source-id: 45f276ef8cbc4704279202f6a20c64889bc1adef
-
- 03 6月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
-
- 02 6月, 2017 1 次提交
-
-
https://github.com/facebook/rocksdb/pull/2082由 Maysam Yabandeh 提交于
Summary: This is a manual commit of this PR: Retire InMemoryEnv in favor of MockEnv #2082 With MockEnv doing the same yet being more mature, InMemoryEnv is redundant. Reviewed By: IslamAbdelRahman Differential Revision: D5162323 fbshipit-source-id: 59fd0082a891dc99cc531e4da9d68bf891eae3f5
-
- 01 6月, 2017 2 次提交
-
-
由 Tamir Duberstein 提交于
Summary: This fixes a compilation failure on Linux when the system libc is not glibc. jemalloc's configure script incorrectly assumes that glibc is always used on Linux systems, producing glibc-style signatures; when the system libc is e.g. musl, the following error is observed: ``` [ 0%] Building CXX object CMakeFiles/rocksdb.dir/db/db_impl.cc.o In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/table/block.h:19:0, from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:77: /x-tools/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot/usr/include/malloc.h:19:8: error: declaration of 'size_t malloc_usable_size(void*)' has a different exception specifier size_t malloc_usable_size(void *); ^~~~~~~~~~~~~~~~~~ In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:20:0: /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:78:33: note: from previous declaration 'size_t malloc_usable_size(void*) throw ()' # define je_malloc_usable_size malloc_usable_size ^ /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:239:41: note: in expansion of macro 'je_malloc_usable_size' JEMALLOC_EXPORT size_t JEMALLOC_NOTHROW je_malloc_usable_size( ^~~~~~~~~~~~~~~~~~~~~ CMakeFiles/rocksdb.dir/build.make:350: recipe for target 'CMakeFiles/rocksdb.dir/db/db_impl.cc.o' failed ``` This works around the issue by rearranging the sources such that jemalloc's headers are never in the same scope as the system's malloc header. The jemalloc issue has been reported as well, see: https://github.com/jemalloc/jemalloc/issues/778. cc tschottdorf Closes https://github.com/facebook/rocksdb/pull/2188 Differential Revision: D5163048 Pulled By: siying fbshipit-source-id: c553125458892def175c1be5682b0330d80b2a0d
-
由 Yi Wu 提交于
Summary: Blob db rely on base db returning sequence number through write batch after DB::Write(). However after recent changes to the write path, DB::Writ()e no longer return sequence number in some cases. Fixing it by have WriteBatchInternal::InsertInto() always encode sequence number into write batch. Stacking on #2375. Closes https://github.com/facebook/rocksdb/pull/2385 Differential Revision: D5148358 Pulled By: yiwu-arbug fbshipit-source-id: 8bda0aa07b9334ed03ed381548b39d167dc20c33
-
- 31 5月, 2017 2 次提交
-
-
由 Yi Wu 提交于
Summary: Re-enable blob_db_test with some update: * Commented out delay at the end of GC tests. Will update the logic later with sync point to properly trigger GC. * Added some helper functions. Also update make files to include blob_dump tool. Closes https://github.com/facebook/rocksdb/pull/2375 Differential Revision: D5133793 Pulled By: yiwu-arbug fbshipit-source-id: 95470b26d0c1f9592ba4b7637e027fdd263f425c
-
由 Tamir Duberstein 提交于
Summary: yiwu-arbug see individual commits. Closes https://github.com/facebook/rocksdb/pull/2318 Differential Revision: D5141520 Pulled By: yiwu-arbug fbshipit-source-id: 7987c92ab4461eef36afce5a133d3a0ee0c96300
-
- 17 5月, 2017 1 次提交
-
-
由 Nikhil Benesch 提交于
Summary: This brings CMake builds further in line with builds that go through the normal Makefile. Closes https://github.com/facebook/rocksdb/pull/2300 Differential Revision: D5064631 Pulled By: yiwu-arbug fbshipit-source-id: 7b2b2d5299f575f87badcf590cc95e040f14d52d
-
- 16 5月, 2017 2 次提交
-
-
由 Nikhil Benesch 提交于
Summary: We've had a couple CockroachDB users fail to build RocksDB on exotic platforms, so I figured I'd try my hand at solving these issues upstream. The problems stem from a) `USE_SSE=1` being too aggressive about turning on SSE4.2, even on toolchains that don't support SSE4.2 and b) RocksDB attempting to detect support for thread-local storage based on OS, even though it can vary by compiler on the same OS. See the individual commit messages for details. Regarding SSE support, this PR should change virtually nothing for non-CMake based builds. `make`, `PORTABLE=1 make`, `USE_SSE=1 make`, and `PORTABLE=1 USE_SSE=1 make` function exactly as before, except that SSE support will be automatically disabled when a simple SSE4.2-using test program fails to compile, as it does on OpenBSD. (OpenBSD's ports GCC supports SSE4.2, but its binutils do not, so `__SSE_4_2__` is defined but an SSE4.2-using program will fail to assemble.) A warning is emitted in this case. The CMake build is modified to support the same set of options, except that `USE_SSE` is spelled `FORCE_SSE42` because `USE_SSE` is rather useless now that we can automatically detect SSE support, and I figure changing options in the CMake build is less disruptive than changing the non-CMake build. I've tested these changes on all the platforms I can get my hands on (macOS, Windows MSVC, Windows MinGW, and OpenBSD) and it all works splendidly. Let me know if there's anything you object to—I obviously don't mean to break any of your build pipelines in the process of fixing ours downstream. Closes https://github.com/facebook/rocksdb/pull/2199 Differential Revision: D5054042 Pulled By: yiwu-arbug fbshipit-source-id: 938e1fc665c049c02ae15698e1409155b8e72171
-
由 Yi Wu 提交于
Summary: snprintf is in <stdio.h> and not in namespace std. Closes https://github.com/facebook/rocksdb/pull/2287 Reviewed By: anirbanr-fb Differential Revision: D5054752 Pulled By: yiwu-arbug fbshipit-source-id: 356807ec38f3c7d95951cdb41f31a3d3ae0714d4
-
- 13 5月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: - Introduced an include/ file dedicated to db-related debug functions to avoid making db.h more complex - Added debugging function, `GetAllKeyVersions()`, to return a listing of internal data for a range of user keys. The new `struct KeyVersion` exposes data similar to internal key without exposing any internal type. - Migrated the "ldb idump" subcommand to use this function - The API takes an inclusive-exclusive range to match behavior of "ldb idump". This will be quite annoying for users who want to query a single user key's versions :(. Closes https://github.com/facebook/rocksdb/pull/2232 Differential Revision: D4976007 Pulled By: ajkr fbshipit-source-id: cab375da53a7595d6575af2b7e3b776aa3ad793e
-
- 12 5月, 2017 1 次提交
-
-
由 Yi Wu 提交于
Summary: Some of the file from #2269 didn't add to CMake file. Adding them to fix window build. Closes https://github.com/facebook/rocksdb/pull/2276 Differential Revision: D5043487 Pulled By: yiwu-arbug fbshipit-source-id: 4eba853e9d92574353abce21d77d30e47ce43d3d
-
- 11 5月, 2017 1 次提交
-
-
由 Anirban Rahut 提交于
Summary: The final pull request for Blob Storage. Closes https://github.com/facebook/rocksdb/pull/2269 Differential Revision: D5033189 Pulled By: yiwu-arbug fbshipit-source-id: 6356b683ccd58cbf38a1dc55e2ea400feecd5d06
-
- 06 5月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: - downcase includes for case-sensitive filesystems - give targets the same name (librocksdb) on all platforms With this patch it is possible to cross-compile RocksDB for Windows from a Linux host using mingw. cc yuslepukhin orgads Closes https://github.com/facebook/rocksdb/pull/2107 Differential Revision: D4849784 Pulled By: siying fbshipit-source-id: ad26ed6b4d393851aa6551e6aa4201faba82ef60
-
- 27 4月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: I'm going to add more DB tests for statistics as currently we have very few. I started a file dedicated to this purpose and moved the existing stats-specific tests there. Closes https://github.com/facebook/rocksdb/pull/2211 Differential Revision: D4951558 Pulled By: ajkr fbshipit-source-id: 05d11c35079c40ecabdfd2cf5556ccb761f694a4
-
- 25 4月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: These code paths forked when checkpoint was introduced by copy/pasting the core backup logic. Over time they diverged and bug fixes were sometimes applied to one but not the other (like fix to include all relevant WALs for 2PC), or it required extra effort to fix both (like fix to forge CURRENT file). This diff reunites the code paths by extracting the core logic into a function, CreateCustomCheckpoint(), that is customizable via callbacks to implement both checkpoint and backup. Related changes: - flush_before_backup is now forcibly enabled when 2PC is enabled - Extracted CheckpointImpl class definition into a header file. This is so the function, CreateCustomCheckpoint(), can be called by internal rocksdb code but not exposed to users. - Implemented more functions in DummyDB/DummyLogFile (in backupable_db_test.cc) that are used by CreateCustomCheckpoint(). Closes https://github.com/facebook/rocksdb/pull/1932 Differential Revision: D4622986 Pulled By: ajkr fbshipit-source-id: 157723884236ee3999a682673b64f7457a7a0d87
-
- 22 4月, 2017 2 次提交
-
-
由 Tomas Kolda 提交于
Summary: Replacement of #2147 The change was squashed due to a lot of conflicts. Closes https://github.com/facebook/rocksdb/pull/2194 Differential Revision: D4929799 Pulled By: siying fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581
-
由 Nick Anderson 提交于
Summary: In the off chance you have a `Microsoft.PowerShell_profile.ps1` profile the `execute_process` will load one's profile during running `cmake`, which will cause garbage output to be written to the `build_version.cc` file. If you add in a `-noprofile` flag, or you suppress output in your pshell profile, this is mitigated. Closes https://github.com/facebook/rocksdb/pull/2168 Differential Revision: D4927003 Pulled By: siying fbshipit-source-id: 83861752d6cf2627dd864eedd2acaa8aa8a6232e
-
- 18 4月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: Seems pretty important. Closes https://github.com/facebook/rocksdb/pull/2161 Differential Revision: D4891097 Pulled By: yiwu-arbug fbshipit-source-id: 16dc0149dc89d426407fabab4a5ba1d6d1562269
-
- 17 4月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2159 Differential Revision: D4894483 Pulled By: yiwu-arbug fbshipit-source-id: 607e17e8ef5d30dce02c27be9d1de7d9f823b4ae
-
- 10 4月, 2017 1 次提交
-
-
由 Willem Jan Withagen 提交于
Summary: The compiler error: ``` /home/jenkins/workspace/ceph-master/src/rocksdb/db/db_impl.cc:20:10: fatal error: 'jemalloc/jemalloc.h' file not found ^ 1 error generated. ``` But is does compile with the `WITH_JEMALLOC` set. So ignore all the other settings. Closes https://github.com/facebook/rocksdb/pull/2118 Differential Revision: D4858387 Pulled By: yiwu-arbug fbshipit-source-id: 05b982969dcab53669a73a903641e71641c714e7
-
- 07 4月, 2017 4 次提交
-
-
由 Siying Dong 提交于
Summary: 1. Move universal compaction picker to separate files compaction_picker_universal.cc and compaction_picker_universal.h. 2. Rename some functions to make the code easier to understand. 3. Move leveled compaction picking code to a dedicated class, so that we we don't need to pass some common variable around when calling functions. It also allowed us to break down LevelCompactionPicker::PickCompaction() to smaller functions. Closes https://github.com/facebook/rocksdb/pull/2100 Differential Revision: D4845948 Pulled By: siying fbshipit-source-id: efa0ab4
-
由 Sagar Vemuri 提交于
Summary: This is an effort to club all string related utility functions into one common place, in string_util, so that it is easier for everyone to know what string processing functions are available. Right now they seem to be spread out across multiple modules, like logging and options_helper. Check the sub-commits for easier reviewing. Closes https://github.com/facebook/rocksdb/pull/2094 Differential Revision: D4837730 Pulled By: sagar0 fbshipit-source-id: 344278a
-
由 Yi Wu 提交于
Summary: Move memtable related files into memtable directory. Closes https://github.com/facebook/rocksdb/pull/2087 Differential Revision: D4829242 Pulled By: yiwu-arbug fbshipit-source-id: ca70ab6
-
由 Tamir Duberstein 提交于
Summary: siying this is a resubmission of #2081 with the 4th commit fixed. From that commit message: > Note that the previous use of quotes in PLATFORM_{CC,CXX}FLAGS was incorrect and caused GCC to produce the incorrect define: > > #define ROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE 1 > > This was the cause of the Linux build failure on the previous version of this change. I've tested this locally, and the Linux build succeeds now. Closes https://github.com/facebook/rocksdb/pull/2097 Differential Revision: D4839964 Pulled By: siying fbshipit-source-id: cc51322
-
- 06 4月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: Move some files under util/ to new directories env/, monitoring/ options/ and cache/ Closes https://github.com/facebook/rocksdb/pull/2090 Differential Revision: D4833681 Pulled By: siying fbshipit-source-id: 2fd8bef
-
由 Siying Dong 提交于
Summary: db_impl.cc is too large to manage. Divide db_impl.cc into db/db_impl.cc, db/db_impl_compaction_flush.cc, db/db_impl_files.cc, db/db_impl_open.cc and db/db_impl_write.cc. Closes https://github.com/facebook/rocksdb/pull/2095 Differential Revision: D4838188 Pulled By: siying fbshipit-source-id: c5f3059
-
- 05 4月, 2017 3 次提交
-
-
由 Siying Dong 提交于
fbshipit-source-id: 43b4529
-
由 Tamir Duberstein 提交于
Summary: See individual commits. yuslepukhin siying Closes https://github.com/facebook/rocksdb/pull/2081 Differential Revision: D4824639 Pulled By: IslamAbdelRahman fbshipit-source-id: 2fc2b00
-
由 Andrew Kryczka 提交于
Summary: I've needed Env timing measurements a few times now, so finally built something for it. Closes https://github.com/facebook/rocksdb/pull/2073 Differential Revision: D4811231 Pulled By: ajkr fbshipit-source-id: 218a249
-
- 04 4月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: It is confusing to have auto_roll_logger to stay under db/, which has nothing to do with database. Move filename together as it is a dependency. Closes https://github.com/facebook/rocksdb/pull/2080 Differential Revision: D4821141 Pulled By: siying fbshipit-source-id: ca7d768
-
由 Tamir Duberstein 提交于
Summary: yuslepukhin See individual commits. Closes https://github.com/facebook/rocksdb/pull/2051 Differential Revision: D4819697 Pulled By: siying fbshipit-source-id: 75c1de4
-
- 31 3月, 2017 1 次提交
-
-
由 Orgad Shaneh 提交于
Summary: There still are many warnings (most of them about invalid printf format for long long), but it builds if FAIL_ON_WARNINGS is disabled. Closes https://github.com/facebook/rocksdb/pull/2052 Differential Revision: D4807355 Pulled By: siying fbshipit-source-id: ef03786
-
- 30 3月, 2017 1 次提交
-
-
由 Min Wei 提交于
Summary: Currently the fast crc32 path is not enabled on Windows. I am trying to enable it here, hopefully, with the minimum impact to the existing code structure. Closes https://github.com/facebook/rocksdb/pull/2033 Differential Revision: D4770635 Pulled By: siying fbshipit-source-id: 676f8b8
-
- 04 3月, 2017 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: This is the first split of https://github.com/facebook/rocksdb/pull/1891 and will be needed for the upcoming partitioned filter patch. Closes https://github.com/facebook/rocksdb/pull/1949 Differential Revision: D4652152 Pulled By: maysamyabandeh fbshipit-source-id: 9801778
-
- 01 3月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Separate the platform dependent tests from external_sst_file_test. Only those tests need to run on platforms like OSX Closes https://github.com/facebook/rocksdb/pull/1923 Differential Revision: D4622461 Pulled By: siying fbshipit-source-id: d2d6f04
-