1. 24 4月, 2020 1 次提交
  2. 18 4月, 2020 1 次提交
    • Y
      Add IsDirectory() to Env and FS (#6711) · 243852ec
      Yanqin Jin 提交于
      Summary:
      IsDirectory() is a common API to check whether a path is a regular file or
      directory.
      POSIX: call stat() and use S_ISDIR(st_mode)
      Windows: PathIsDirectoryA() and PathIsDirectoryW()
      HDFS: FileSystem.IsDirectory()
      Java: File.IsDirectory()
      ...
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6711
      
      Test Plan: make check
      
      Reviewed By: anand1976
      
      Differential Revision: D21053520
      
      Pulled By: riversand963
      
      fbshipit-source-id: 680aadfd8ce982b63689190cf31b3145d5a89e27
      243852ec
  3. 17 4月, 2020 1 次提交
  4. 02 4月, 2020 1 次提交
  5. 30 3月, 2020 1 次提交
  6. 28 3月, 2020 1 次提交
    • Z
      Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487) · 42468881
      Zhichao Cao 提交于
      Summary:
      In the current code base, we use Status to get and store the returned status from the call. Specifically, for IO related functions, the current Status cannot reflect the IO Error details such as error scope, error retryable attribute, and others. With the implementation of https://github.com/facebook/rocksdb/issues/5761, we have the new Wrapper for IO, which returns IOStatus instead of Status. However, the IOStatus is purged at the lower level of write path and transferred to Status.
      
      The first job of this PR is to pass the IOStatus to the write path (flush, WAL write, and Compaction). The second job is to identify the Retryable IO Error as HardError, and set the bg_error_ as HardError. In this case, the DB Instance becomes read only. User is informed of the Status and need to take actions to deal with it (e.g., call db->Resume()).
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6487
      
      Test Plan: Added the testing case to error_handler_fs_test. Pass make asan_check
      
      Reviewed By: anand1976
      
      Differential Revision: D20685017
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: ff85f042896243abcd6ef37877834e26f36b6eb0
      42468881
  7. 24 3月, 2020 1 次提交
    • A
      Simplify migration to FileSystem API (#6552) · a9d168cf
      anand76 提交于
      Summary:
      The current Env/FileSystem API separation has a couple of issues -
      1. It requires the user to specify 2 options - ```Options::env``` and ```Options::file_system``` - which means they have to make code changes to benefit from the new APIs. Furthermore, there is a risk of accessing the same APIs in two different ways, through Env in the old way and through FileSystem in the new way. The two may not always match, for example, if env is ```PosixEnv``` and FileSystem is a custom implementation. Any stray RocksDB calls to env will use the ```PosixEnv``` implementation rather than the file_system implementation.
      2. There needs to be a simple way for the FileSystem developer to instantiate an Env for backward compatibility purposes.
      
      This PR solves the above issues and simplifies the migration in the following ways -
      1. Embed a shared_ptr to the ```FileSystem``` in the ```Env```, and remove ```Options::file_system``` as a configurable option. This way, no code changes will be required in application code to benefit from the new API. The default Env constructor uses a ```LegacyFileSystemWrapper``` as the embedded ```FileSystem```.
      1a. - This also makes it more robust by ensuring that even if RocksDB
        has some stray calls to Env APIs rather than FileSystem, they will go
        through the same object and thus there is no risk of getting out of
        sync.
      2. Provide a ```NewCompositeEnv()``` API that can be used to construct a
      PosixEnv with a custom FileSystem implementation. This eliminates an
      indirection to call Env APIs, and relieves the FileSystem developer of
      the burden of having to implement wrappers for the Env APIs.
      3. Add a couple of missing FileSystem APIs - ```SanitizeEnvOptions()``` and
      ```NewLogger()```
      
      Tests:
      1. New unit tests
      2. make check and make asan_check
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6552
      
      Reviewed By: riversand963
      
      Differential Revision: D20592038
      
      Pulled By: anand1976
      
      fbshipit-source-id: c3801ad4153f96d21d5a3ae26c92ba454d1bf1f7
      a9d168cf
  8. 21 3月, 2020 2 次提交
    • Y
      Attempt to recover from db with missing table files (#6334) · fb09ef05
      Yanqin Jin 提交于
      Summary:
      There are situations when RocksDB tries to recover, but the db is in an inconsistent state due to SST files referenced in the MANIFEST being missing. In this case, previous RocksDB will just fail the recovery and return a non-ok status.
      This PR enables another possibility. During recovery, RocksDB checks possible MANIFEST files, and try to recover to the most recent state without missing table file. `VersionSet::Recover()` applies version edits incrementally and "materializes" a version only when this version does not reference any missing table file. After processing the entire MANIFEST, the version created last will be the latest version.
      `DBImpl::Recover()` calls `VersionSet::Recover()`. Afterwards, WAL replay will *not* be performed.
      To use this capability, set `options.best_efforts_recovery = true` when opening the db. Best-efforts recovery is currently incompatible with atomic flush.
      
      Test plan (on devserver):
      ```
      $make check
      $COMPILE_WITH_ASAN=1 make all && make check
      ```
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6334
      
      Reviewed By: anand1976
      
      Differential Revision: D19778960
      
      Pulled By: riversand963
      
      fbshipit-source-id: c27ea80f29bc952e7d3311ecf5ee9c54393b40a8
      fb09ef05
    • C
      Get block size only in direct IO mode (#6522) · 5fd152b7
      Cheng Chang 提交于
      Summary:
      When `use_direct_reads` and `use_direct_writes` are `false`, `logical_sector_size_` inside various `*File` implementations are not actually used, so `GetLogicalBlockSize` does not necessarily need to be called for `logical_sector_size_`, just set a default page size.
      
      This is a follow up PR for https://github.com/facebook/rocksdb/pull/6457.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6522
      
      Test Plan: make check
      
      Reviewed By: siying
      
      Differential Revision: D20408885
      
      Pulled By: cheng-chang
      
      fbshipit-source-id: f2d3808f41265237e7fa2c0be9f084f8fa97fe3d
      5fd152b7
  9. 12 3月, 2020 2 次提交
    • C
      Cache result of GetLogicalBufferSize in Linux (#6457) · 2d9efc9a
      Cheng Chang 提交于
      Summary:
      In Linux, when reopening DB with many SST files, profiling shows that 100% system cpu time spent for a couple of seconds for `GetLogicalBufferSize`. This slows down MyRocks' recovery time when site is down.
      
      This PR introduces two new APIs:
      1. `Env::RegisterDbPaths` and `Env::UnregisterDbPaths` lets `DB` tell the env when it starts or stops using its database directories . The `PosixFileSystem` takes this opportunity to set up a cache from database directories to the corresponding logical block sizes.
      2. `LogicalBlockSizeCache` is defined only for OS_LINUX to cache the logical block sizes.
      
      Other modifications:
      1. rename `logical buffer size` to `logical block size` to be consistent with Linux terms.
      2. declare `GetLogicalBlockSize` in `PosixHelper` to expose it to `PosixFileSystem`.
      3. change the functions `IOError` and `IOStatus` in `env/io_posix.h` to have external linkage since they are used in other translation units too.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6457
      
      Test Plan:
      1. A new unit test is added for `LogicalBlockSizeCache` in `env/io_posix_test.cc`.
      2. A new integration test is added for `DB` operations related to the cache in `db/db_logical_block_size_cache_test.cc`.
      
      `make check`
      
      Differential Revision: D20131243
      
      Pulled By: cheng-chang
      
      fbshipit-source-id: 3077c50f8065c0bffb544d8f49fb10bba9408d04
      2d9efc9a
    • S
      Include more information in file lock failure (#6507) · 331e6199
      sdong 提交于
      Summary:
      When users fail to open a DB with file lock failure, it is sometimes hard for users to debug. We now include the time the lock is acquired and the thread ID that acquired the lock, to help users debug problems like this. Default Env's thread ID is used.
      
      Since type of lockedFiles is changed, rename it to follow naming convention too.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6507
      
      Test Plan: Add a unit test and improve an existing test to validate the case.
      
      Differential Revision: D20378333
      
      fbshipit-source-id: 312fe0e9733fd1d1e9969c321b90ce523cf4708a
      331e6199
  10. 03 3月, 2020 1 次提交
  11. 29 2月, 2020 2 次提交
  12. 23 2月, 2020 1 次提交
  13. 22 2月, 2020 1 次提交
    • S
      Handle io_uring partial results (#6441) · 942eaba0
      sdong 提交于
      Summary:
      The logic that handles io_uring partial results was wrong. Fix the logic by putting it into a queue and continue reading.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6441
      
      Test Plan: Make sure this patch fixes the application test case where the bug was discovered; in env_test, add a unit test that simulates partial results and make sure the results are still correct.
      
      Differential Revision: D20018616
      
      fbshipit-source-id: 5398a7e34d74c26d52aa69dfd604e93e95d99c62
      942eaba0
  14. 21 2月, 2020 1 次提交
    • S
      Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) · fdf882de
      sdong 提交于
      Summary:
      When dynamically linking two binaries together, different builds of RocksDB from two sources might cause errors. To provide a tool for user to solve the problem, the RocksDB namespace is changed to a flag which can be overridden in build time.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6433
      
      Test Plan: Build release, all and jtest. Try to build with ROCKSDB_NAMESPACE with another flag.
      
      Differential Revision: D19977691
      
      fbshipit-source-id: aa7f2d0972e1c31d75339ac48478f34f6cfcfb3e
      fdf882de
  15. 14 2月, 2020 1 次提交
    • C
      Fix flaky test DecreaseNumBgThreads (#6393) · 46516778
      Cheng Chang 提交于
      Summary:
      The DecreaseNumBgThreads test keeps failing on Windows in AppVeyor.
      It fails because it depends on a timed wait for the tasks to be dequeued from the threadpool's internal queue, but within the specified time, the task might have not been scheduled onto the newly created threads.
      https://github.com/facebook/rocksdb/pull/6232 tries to fix this by waiting for longer time to let the threads scheduled.
      This PR tries to fix this by replacing the timed wait with a synchronization on the task's internal conditional variable.
      When the number of threads increases, instead of guessing the time needed for the task to be scheduled, it directly blocks on the conditional variable until the task starts running.
      But when thread number is reduced, it still does a timed wait, but this does not lead to the flakiness now, will try to remove these timed waits in a future PR.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6393
      
      Test Plan: Wait to see whether AppVeyor tests pass.
      
      Differential Revision: D19890928
      
      Pulled By: cheng-chang
      
      fbshipit-source-id: 4e56e4addf625c98c0876e62d9d57a6f0a156f76
      46516778
  16. 05 2月, 2020 1 次提交
    • S
      Consolidate ReadFileToString() (#6366) · 3a073234
      sdong 提交于
      Summary:
      It's a minor refactoring. We have two ReadFileToString() but they are very similar. Make the one with Env argument calls the one with FS argument instead.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6366
      
      Test Plan: Run all existing tests
      
      Differential Revision: D19712332
      
      fbshipit-source-id: 5ae6fabf6355938690d95cda52afd1f39e0a7823
      3a073234
  17. 09 1月, 2020 1 次提交
    • S
      Fix some shadow warning (#6242) · 39410bcb
      sdong 提交于
      Summary:
      Some shadow warning shows up when using gcc 4.8. An example:
      
      ./utilities/blob_db/blob_compaction_filter.h: In constructor ‘rocksdb::blob_db::BlobIndexCompactionFilterFactoryBase::BlobIndexCompactionFilterFactoryBase(rocksdb::blob_db::lobDBImpl*, rocksdb::Env*, rocksdb::Statistics*)’:
      ./utilities/blob_db/blob_compaction_filter.h:121:7: error: declaration of ‘blob_db_impl’ shadows a member of 'this' [-Werror=shadow]
             : blob_db_impl_(blob_db_impl), env_(_env), statistics_(_statistics) {}
             ^
      
      Fix them.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6242
      
      Test Plan: Build and see the warnings go away.
      
      Differential Revision: D19217789
      
      fbshipit-source-id: 8ef631941f23dab47a388e060adec24b72efd65e
      39410bcb
  18. 15 12月, 2019 1 次提交
  19. 14 12月, 2019 1 次提交
    • A
      Introduce a new storage specific Env API (#5761) · afa2420c
      anand76 提交于
      Summary:
      The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
      
      This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
      
      The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
      
      This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
      
      The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
      
      Differential Revision: D18868376
      
      Pulled By: anand1976
      
      fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
      afa2420c
  20. 10 12月, 2019 1 次提交
    • S
      Fix an asan warning caused by the recent io_uring change (#6135) · d1ae2c3f
      sdong 提交于
      Summary:
      ASAN reports:
      
      internal_repo_rocksdb/repo:db_test - MultiThreaded/MultiThreadedDBTest.MultiThreaded/43: fatal
      ==2692739==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6130000500ca at pc 0x0000006be780 bp 0x7efef85ccd20 sp 0x7efef85cc4d0
      [CONTEXT] === How to use this, how to get the raw stack trace, and more: fburl.com/ASAN ===
      [CONTEXT] READ of size 331 at 0x6130000500ca thread T195
      [CONTEXT]      #0 db_test_bin+0x6be77f                     __interceptor_strlen.part.35
      [CONTEXT]      https://github.com/facebook/rocksdb/issues/1 internal_repo_rocksdb/repo/include/rocksdb/slice.h:55 rocksdb::Slice::Slice(char const*)
      [CONTEXT]      https://github.com/facebook/rocksdb/issues/2 internal_repo_rocksdb/repo/env/io_posix.cc:522 rocksdb::PosixRandomAccessFile::MultiRead(rocksdb::ReadRequest*, unsigned long)
      
      I looked at env/io_posix.cc:522 but don't see a reason why the line needs to be there at all, because it is not used before overwritten. So it must be a line that is put there as a bug. Remove it.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6135
      
      Test Plan: Rerun the same test which passes after the fix. Run all the tests and make sure they all pass.
      
      Differential Revision: D18880251
      
      fbshipit-source-id: 3b84ac6a05b67b529c4202e0ceb4c047460f44f2
      d1ae2c3f
  21. 09 12月, 2019 1 次提交
    • S
      Break db_stress_tool.cc to a list of source files (#6134) · 7d79b326
      sdong 提交于
      Summary:
      db_stress_tool.cc now is a giant file. In order to main it easier to improve and maintain, break it down to multiple source files.
      Most classes are turned into their own files. Separate .h and .cc files are created for gflag definiations. Another .h and .cc files are created for some common functions. Some test execution logic that is only loosely related to class StressTest is moved to db_stress_driver.h and db_stress_driver.cc. All the files are located under db_stress_tool/. The directory name is created as such because if we end it with either stress or test, .gitignore will ignore any file under it and makes it prone to issues in developements.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6134
      
      Test Plan: Build under GCC7 with and without LITE on using GNU Make. Build with GCC 4.8. Build with cmake with -DWITH_TOOL=1
      
      Differential Revision: D18876064
      
      fbshipit-source-id: b25d0a7451840f31ac0f5ebb0068785f783fdf7d
      7d79b326
  22. 08 12月, 2019 1 次提交
    • S
      PosixRandomAccessFile::MultiRead() to use I/O uring if supported (#5881) · e3a82bb9
      sdong 提交于
      Summary:
      Right now, PosixRandomAccessFile::MultiRead() executes read requests in parallel. In this PR, it leverages I/O Uring library to run it in parallel, even when page cache is enabled. This function will fall back if the kernel version doesn't support it.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5881
      
      Test Plan: Run the unit test on a kernel version supporting it and make sure all tests pass, and run a unit test on kernel version supporting it and see it pass. Before merging, will also run stress test and see it passes.
      
      Differential Revision: D17742266
      
      fbshipit-source-id: e05699c925ac04fdb42379456a4e23e4ebcb803a
      e3a82bb9
  23. 15 10月, 2019 1 次提交
  24. 12 10月, 2019 1 次提交
    • A
      Fix block cache ID uniqueness for Windows builds (#5844) · b00761ee
      Andrew Kryczka 提交于
      Summary:
      Since we do not evict a file's blocks from block cache before that file
      is deleted, we require a file's cache ID prefix is both unique and
      non-reusable. However, the Windows functionality we were relying on only
      guaranteed uniqueness. That meant a newly created file could be assigned
      the same cache ID prefix as a deleted file. If the newly created file
      had block offsets matching the deleted file, full cache keys could be
      exactly the same, resulting in obsolete data blocks returned from cache
      when trying to read from the new file.
      
      We noticed this when running on FAT32 where compaction was writing out
      of order keys due to reading obsolete blocks from its input files. The
      functionality is documented as behaving the same on NTFS, although I
      wasn't able to repro it there.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5844
      
      Test Plan:
      we had a reliable repro of out-of-order keys on FAT32 that
      was fixed by this change
      
      Differential Revision: D17752442
      
      fbshipit-source-id: 95d983f9196cf415f269e19293b97341edbf7e00
      b00761ee
  25. 09 10月, 2019 1 次提交
    • Y
      Support custom env in sst_dump (#5845) · 167cdc9f
      Yanqin Jin 提交于
      Summary:
      This PR allows for the creation of custom env when using sst_dump. If
      the user does not set options.env or set options.env to nullptr, then sst_dump
      will automatically try to create a custom env depending on the path to the sst
      file or db directory. In order to use this feature, the user must call
      ObjectRegistry::Register() beforehand.
      
      Test Plan (on devserver):
      ```
      $make all && make check
      ```
      All tests must pass to ensure this change does not break anything.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5845
      
      Differential Revision: D17678038
      
      Pulled By: riversand963
      
      fbshipit-source-id: 58ecb4b3f75246d52b07c4c924a63ee61c1ee626
      167cdc9f
  26. 21 9月, 2019 1 次提交
  27. 17 9月, 2019 1 次提交
    • M
      Charge block cache for cache internal usage (#5797) · 638d2395
      Maysam Yabandeh 提交于
      Summary:
      For our default block cache, each additional entry has extra memory overhead. It include LRUHandle (72 bytes currently) and the cache key (two varint64, file id and offset). The usage is not negligible. For example for block_size=4k, the overhead accounts for an extra 2% memory usage for the cache. The patch charging the cache for the extra usage, reducing untracked memory usage outside block cache. The feature is enabled by default and can be disabled by passing kDontChargeCacheMetadata to the cache constructor.
      This PR builds up on https://github.com/facebook/rocksdb/issues/4258
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5797
      
      Test Plan:
      - Existing tests are updated to either disable the feature when the test has too much dependency on the old way of accounting the usage or increasing the cache capacity to account for the additional charge of metadata.
      - The Usage tests in cache_test.cc are augmented to test the cache usage under kFullChargeCacheMetadata.
      
      Differential Revision: D17396833
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: 7684ccb9f8a40ca595e4f5efcdb03623afea0c6f
      638d2395
  28. 12 9月, 2019 2 次提交
    • S
      Use delete to disable automatic generated methods. (#5009) · 9eb3e1f7
      Shylock Hg 提交于
      Summary:
      Use delete to disable automatic generated methods instead of private, and put the constructor together for more clear.This modification cause the unused field warning, so add unused attribute to disable this warning.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5009
      
      Differential Revision: D17288733
      
      fbshipit-source-id: 8a767ce096f185f1db01bd28fc88fef1cdd921f3
      9eb3e1f7
    • A
      Avoid clock_gettime on pre-10.12 macOS versions (#5570) · 20dd828c
      Andrew Kryczka 提交于
      Summary:
      On older macOS like 10.10 we saw the following compiler error:
      
      ```
      /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb/env/env_posix.cc:845:19:
      error: use of undeclared identifier 'CLOCK_THREAD_CPUTIME_ID'
          clock_gettime(CLOCK_THREAD_CPUTIME_ID, &ts);
                        ^
      ```
      
      According to mac's `man clock_gettime`: "These functions first appeared in Mac
      OSX 10.12". So we should not try to compile it on earlier versions.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5570
      
      Test Plan:
      verified it compiles now on 10.10. Also did some investigation to
      ensure it does not cause regression on macOS 10.12+, although I do not
      have access to such an environment to really test.
      
      Differential Revision: D17322629
      
      Pulled By: riversand963
      
      fbshipit-source-id: e0a412223854f826b4d83e6d15c3739ff4620d7d
      20dd828c
  29. 06 9月, 2019 2 次提交
  30. 17 8月, 2019 1 次提交
  31. 31 7月, 2019 1 次提交
  32. 24 7月, 2019 1 次提交
    • M
      The ObjectRegistry class replaces the Registrar and NewCustomObjects.… (#5293) · cfcf045a
      Mark Rambacher 提交于
      Summary:
      The ObjectRegistry class replaces the Registrar and NewCustomObjects.  Objects are registered with the registry by Type (the class must implement the static const char *Type() method).
      
      This change is necessary for a few reasons:
      - By having a class (rather than static template instances), the class can be passed between compilation units, meaning that objects could be registered and shared from a dynamic library with an executable.
      - By having a class with instances, different units could have different objects registered.  This could be useful if, for example, one Option allowed for a dynamic library and one did not.
      
      When combined with some other PRs (being able to load shared libraries, a Configurable interface to configure objects to/from string), this code will allow objects in external shared libraries to be added to a RocksDB image at run-time, rather than requiring every new extension to be built into the main library and called explicitly by every program.
      
      Test plan (on riversand963's  devserver)
      ```
      $COMPILE_WITH_ASAN=1 make -j32 all && sleep 1 && make check
      ```
      All tests pass.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5293
      
      Differential Revision: D16363396
      
      Pulled By: riversand963
      
      fbshipit-source-id: fbe4acb615bfc11103eef40a0b288845791c0180
      cfcf045a
  33. 10 7月, 2019 1 次提交
  34. 20 6月, 2019 1 次提交
    • S
      Fix AlignedBuffer's usage in Encryption Env (#5396) · 68614a96
      Sagar Vemuri 提交于
      Summary:
      The usage of `AlignedBuffer` in env_encryption.cc writes and reads to/from the AlignedBuffer's internal buffer directly without going through AlignedBuffer's APIs (like `Append` and `Read`), causing encapsulation to break in some cases. The writes are especially problematic as after the data is written to the buffer (directly using either memmove or memcpy), the size of the buffer is not updated ... causing the AlignedBuffer to lose track of the encapsulated buffer's current size.
      Fixed this by updating the buffer size after every write.
      
      Todo for later:
      Add an overloaded method to AlignedBuffer to support a memmove in addition to a memcopy. Encryption env does a memmove, and hence I couldn't switch to using `AlignedBuffer.Append()`.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5396
      
      Test Plan: `make check`
      
      Differential Revision: D15764756
      
      Pulled By: sagar0
      
      fbshipit-source-id: 2e24b52bd3b4b5056c5c1da157f91ddf89370183
      68614a96
  35. 19 6月, 2019 1 次提交