Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 01 9月, 2017 2 次提交
  3. 15 8月, 2017 1 次提交
  5. 11 8月, 2017 2 次提交
  7. 10 8月, 2017 1 次提交
    • D
      Makefile: correct faligned-new test · 1fbad84b
      由 Daniel Black 提交于 
      Summary:
Commit 4f81ab38 has the test wrong.

clang doesn't support a -dumpversion option. By lucky coincidence
clang/gcc --version both place a version number at the same output location
when --verison is passed.

Example output (1st line only).

    $ clang --version
    clang version 3.9.1 (tags/RELEASE_391/final)

    $ gcc --version
    gcc (GCC) 6.4.1 20170727 (Red Hat 6.4.1-1)

During the test of the compiler we ensure that a minimum version is met
as Makefile doesn't support patterns.

Also xcode9 doesn't seem affected by https://github.com/facebook/rocksdb/issues/2672
and also doesn't have "clang" as the first part of its output so the
fix implemented here also is Apple clang friendly.

    $ clang --version
    Apple LLVM version 9.0.0 (clang-900.0.31)
Signed-off-by: NDaniel Black <daniel.black@au.ibm.com>
Closes https://github.com/facebook/rocksdb/pull/2699

Differential Revision: D5600818

Pulled By: yiwu-arbug

fbshipit-source-id: 3b0f2751becb53c1c35468bf29f3f828e7cf2c2a
      1fbad84b
  9. 05 8月, 2017 1 次提交
  11. 04 8月, 2017 1 次提交
  13. 29 7月, 2017 1 次提交
    • S
      Replace dynamic_cast<> · 21696ba5
      由 Siying Dong 提交于 
      Summary:
Replace dynamic_cast<> so that users can choose to build with RTTI off, so that they can save several bytes per object, and get tiny more memory available.
Some nontrivial changes:
1. Add Comparator::GetRootComparator() to get around the internal comparator hack
2. Add the two experiemental functions to DB
3. Add TableFactory::GetOptionString() to avoid unnecessary casting to get the option string
4. Since 3 is done, move the parsing option functions for table factory to table factory files too, to be symmetric.
Closes https://github.com/facebook/rocksdb/pull/2645

Differential Revision: D5502723

Pulled By: siying

fbshipit-source-id: fd13cec5601cf68a554d87bfcf056f2ffa5fbf7c
      21696ba5
  15. 28 7月, 2017 1 次提交
  17. 27 7月, 2017 2 次提交
  19. 22 7月, 2017 1 次提交
    • P
      Cassandra compaction filter for purge expired columns and rows · 534c255c
      由 Pengchao Wang 提交于 
      Summary:
Major changes in this PR:
* Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired)
* Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/*
* Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable.
* Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there.
Closes https://github.com/facebook/rocksdb/pull/2588

Differential Revision: D5430010

Pulled By: wpc

fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
      534c255c
  21. 11 7月, 2017 1 次提交
    • G
      Fix undefined behavior in Hash · 8f927e5f
      由 Giuseppe Ottaviano 提交于 
      Summary:
Instead of ignoring UBSan checks, fix the negative shifts in
Hash(). Also add test to make sure the hash values are stable over
time. The values were computed before this change, so the test also
verifies the correctness of the change.
Closes https://github.com/facebook/rocksdb/pull/2546

Differential Revision: D5386369

Pulled By: yiwu-arbug

fbshipit-source-id: 6de4b44461a544d6222cc5d72d8cda2c0373d17e
      8f927e5f
  23. 28 6月, 2017 1 次提交
    • Y
      Fix TARGETS file tests list · 982cec22
      由 Yi Wu 提交于 
      Summary:
1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them.
2. add blob_db_test
Closes https://github.com/facebook/rocksdb/pull/2506

Differential Revision: D5331517

Pulled By: yiwu-arbug

fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
      982cec22
  25. 27 6月, 2017 1 次提交
    • E
      Encryption at rest support · 51778612
      由 Ewout Prangsma 提交于 
      Summary:
This PR adds support for encrypting data stored by RocksDB when written to disk.

It adds an `EncryptedEnv` override of the `Env` class with matching overrides for sequential&random access files.
The encryption itself is done through a configurable `EncryptionProvider`. This class creates is asked to create `BlockAccessCipherStream` for a file. This is where the actual encryption/decryption is being done.
Currently there is a Counter mode implementation of `BlockAccessCipherStream` with a `ROT13` block cipher (NOTE the `ROT13` is for demo purposes only!!).

The Counter operation mode uses an initial counter & random initialization vector (IV).
Both are created randomly for each file and stored in a 4K (default size) block that is prefixed to that file. The `EncryptedEnv` implementation is such that clients of the `Env` class do not see this prefix (nor data, nor in filesize).
The largest part of the prefix block is also encrypted, and there is room left for implementation specific settings/values/keys in there.

To test the encryption, the `DBTestBase` class has been extended to consider a new environment variable called `ENCRYPTED_ENV`. If set, the test will setup a encrypted instance of the `Env` class to use for all tests.
Typically you would run it like this:

```
ENCRYPTED_ENV=1 make check_some
```

There is also an added test that checks that some data inserted into the database is or is not "visible" on disk. With `ENCRYPTED_ENV` active it must not find plain text strings, with `ENCRYPTED_ENV` unset, it must find the plain text strings.
Closes https://github.com/facebook/rocksdb/pull/2424

Differential Revision: D5322178

Pulled By: sdwilsh

fbshipit-source-id: 253b0a9c2c498cc98f580df7f2623cbf7678a27f
      51778612
  27. 17 6月, 2017 1 次提交
  29. 06 6月, 2017 1 次提交
  31. 03 6月, 2017 1 次提交
    • S
      Improve write buffer manager (and allow the size to be tracked in block cache) · 95b0e89b
      由 Siying Dong 提交于 
      Summary:
Improve write buffer manager in several ways:
1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower.
2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit.
3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable.
Closes https://github.com/facebook/rocksdb/pull/2350

Differential Revision: D5110648

Pulled By: siying

fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
      95b0e89b
  33. 01 6月, 2017 1 次提交
    • Y
      Fixing blob db sequence number handling · ad19eb86
      由 Yi Wu 提交于 
      Summary:
Blob db rely on base db returning sequence number through write batch after DB::Write(). However after recent changes to the write path, DB::Writ()e no longer return sequence number in some cases. Fixing it by have WriteBatchInternal::InsertInto() always encode sequence number into write batch.

Stacking on #2375.
Closes https://github.com/facebook/rocksdb/pull/2385

Differential Revision: D5148358

Pulled By: yiwu-arbug

fbshipit-source-id: 8bda0aa07b9334ed03ed381548b39d167dc20c33
      ad19eb86
  35. 31 5月, 2017 2 次提交
  37. 27 5月, 2017 1 次提交
    • S
      Clean zstd files · 6c456eca
      由 Sagar Vemuri 提交于 
      Summary:
zstd files are downloaded and used as part of JNI build, but are left behind even after doing a `make clean`. This PR updates the `clean` target to remove these zstd files as well.
Closes https://github.com/facebook/rocksdb/pull/2365

Differential Revision: D5123537

Pulled By: sagar0

fbshipit-source-id: a8f355da5ba961aa89d5852e35751ffc35de03ea
      6c456eca
  39. 24 5月, 2017 1 次提交
  41. 19 5月, 2017 1 次提交
  43. 18 5月, 2017 2 次提交
  45. 16 5月, 2017 2 次提交
  47. 13 5月, 2017 1 次提交
    • A
      Facility for cross-building RocksJava using Docker · a5cc7ece
      由 Adam Retter 提交于 
      Summary:
As an alternative to Vagrant, we can now also use Docker to cross-build RocksDB. The advantages are:

1. The Docker images are fixed; they include all the latest updates and build tools.
2. The Vagrant image, required scripts that ran for every build that would update CentOS and install the buildtools. This lead to slow repeatable builds, we don't need to do this with Docker as they are already in the provided images.

The Docker images I have used have their Docker build files here: https://github.com/evolvedbinary/docker-rocksjava and the images themselves are available from Docker hub: https://hub.docker.com/r/evolvedbinary/rocksjava/

I have added the following targets to the `Makefile`:
1. `rocksdbjavastaticreleasedocker` this uses Docker to perform the cross-builds. It is basically the Docker version of the existing Vagrant `rocksdbjavastaticrelease` target.
2. `rocksdbjavastaticpublishdocker` delegates to `rocksdbjavastaticreleasedocker` and then `rocksdbjavastaticpublishcentral` to upload the artiacts to Maven Central. Equivalent to the existing Vagrant target: `rocksdbjavastaticpublish`
Closes https://github.com/facebook/rocksdb/pull/2278

Differential Revision: D5048206

Pulled By: yiwu-arbug

fbshipit-source-id: 78fa96ef9d966fe09638ed01de282cd4e31961a9
      a5cc7ece
  49. 12 5月, 2017 1 次提交
  51. 11 5月, 2017 2 次提交
    • A
      Blob storage pr · d85ff495
      由 Anirban Rahut 提交于 
      Summary:
The final pull request for Blob Storage.
Closes https://github.com/facebook/rocksdb/pull/2269

Differential Revision: D5033189

Pulled By: yiwu-arbug

fbshipit-source-id: 6356b683ccd58cbf38a1dc55e2ea400feecd5d06
      d85ff495
    • L
      Add NO_UPDATE_BUILD_VERSION option to makefile · 0f559abd
      由 Lovro Puzar 提交于 
      Summary:
When building rocksdb in fbcode using `make`, util/build_version.cc is always updated (gitignore/hgignore doesn't apply because the file is already checked into fbcode).  To use the rocksdb makefile from our own makefile, I would like an option to prevent the metadata update, which is of no value for us.
Closes https://github.com/facebook/rocksdb/pull/2264

Differential Revision: D5037846

Pulled By: siying

fbshipit-source-id: 9fa005725c5ecb31d9cbe2e738cbee209591f08a
      0f559abd
  53. 27 4月, 2017 1 次提交
  55. 22 4月, 2017 1 次提交
  57. 17 4月, 2017 2 次提交
    • J
      enable O2 optimization for lz4 · e67f0adf
      由 Jay Lee 提交于 
      Summary: Closes https://github.com/facebook/rocksdb/pull/2164

Differential Revision: D4897389

Pulled By: yiwu-arbug

fbshipit-source-id: fac15374ae7fef1ece70fd2b9018f2451f3c2f7c
      e67f0adf
    • T
      Separate compile and link for shared library · 7d5f5aa9
      由 Tudor Bosman 提交于 
      Summary:
Previously, the shared library (make shared_lib) was built with only one
compile line, compiling all .cc files and linking the shared library in
one step. That step would often take 10+ minutes on one machine, and
could not take advantage of multiple CPUs (it's only one invocation of
the compiler).

This commit changes the shared_lib build to compile .o files
individually (placing the resulting .o files in the directory
shared-objects) and then link them into the shared library at the end,
similarly to how the java static build (jls) does it.

Tested by making sure that both static and shared libraries work, and by
making sure that "make clean" cleans up the shared-objects directory.
Closes https://github.com/facebook/rocksdb/pull/2165

Differential Revision: D4897121

Pulled By: yiwu-arbug

fbshipit-source-id: 9811e043d1c01e10503593f3489d186c786ee7d7
      7d5f5aa9
  59. 12 4月, 2017 1 次提交
    • S
      Add ROCKSDB_JAVA_NO_COMPRESSION flag · 6257837d
      由 Siying Dong 提交于 
      Summary:
In some CI test environment, compression libraries can't be successfully built. It still helps to build RocksDB there. Provide such an option to skip to download and build compression libraries.
Closes https://github.com/facebook/rocksdb/pull/2135

Differential Revision: D4872617

Pulled By: siying

fbshipit-source-id: bb21ac373bc62a2528cdf1ca4547e05fcae86214
      6257837d
  61. 07 4月, 2017 2 次提交
    • Y
      Move memtable related files into memtable directory · df6f5a37
      由 Yi Wu 提交于 
      Summary:
Move memtable related files into memtable directory.
Closes https://github.com/facebook/rocksdb/pull/2087

Differential Revision: D4829242

Pulled By: yiwu-arbug

fbshipit-source-id: ca70ab6
      df6f5a37
    • T
      CMake: more MinGW fixes · 107c5f6a
      由 Tamir Duberstein 提交于 
      Summary:
siying this is a resubmission of #2081 with the 4th commit fixed. From that commit message:

> Note that the previous use of quotes in PLATFORM_{CC,CXX}FLAGS was
incorrect and caused GCC to produce the incorrect define:
>
>  #define ROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE 1
>
> This was the cause of the Linux build failure on the previous version
of this change.

I've tested this locally, and the Linux build succeeds now.
Closes https://github.com/facebook/rocksdb/pull/2097

Differential Revision: D4839964

Pulled By: siying

fbshipit-source-id: cc51322
      107c5f6a