1. 16 9月, 2020 1 次提交
    • M
      Changes to EncryptedEnv public API (#7279) · 67bd5401
      mrambacher 提交于
      Summary:
      Cleaned up the public API to use the EncryptedEnv.  This change will allow providers to be developed and added to the system easier in the future.  It will also allow better integration in the future with the OPTIONS file.
      
      - The internal classes were moved out of the public API into an internal "env_encryption_ctr.h" header.  Short-cut constructors were added to provide the original API functionality.
      - The APIs to the constructors were changed to take shared_ptr, rather than raw pointers or references to allow better memory management and alternative implementations.
      - CreateFromString methods were added to allow future expansion to other provider and cipher implementations through a standard API.
      
      Additionally, there was a code duplication in the NewXXXFile methods.  This common code was moved under a templatized function.
      
      A first-pass at structuring the code was made to potentially allow multiple EncryptionProviders in a single EncryptedEnv.  The idea was that different providers may use different cipher keys or different versions/algorithms.  The EncryptedEnv should have some means of picking different providers based on information.  The groundwork was started for this (the use of the provider_ member variable was localized) but the work has not been completed.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7279
      
      Reviewed By: jay-zhuang
      
      Differential Revision: D23709440
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: 0e845fff0e03a52603eb9672b4ade32d063ff2f2
      67bd5401
  2. 15 9月, 2020 7 次提交
    • L
      Integrate blob file writing with the flush logic (#7345) · b0e78341
      Levi Tamasi 提交于
      Summary:
      The patch adds support for writing blob files during flush by integrating
      `BlobFileBuilder` with the flush logic, most importantly, `BuildTable` and
      `CompactionIterator`. If `enable_blob_files` is set, large values are extracted
      to blob files and replaced with references. The resulting blob files are then
      logged to the MANIFEST as part of the flush job's `VersionEdit` and
      added to the `Version`, similarly to table files. Errors related to writing
      blob files fail the flush, and any blob files written by such jobs are immediately
      deleted (again, similarly to how SST files are handled). In addition, the patch
      extends the logging and statistics around flushes to account for the presence
      of blob files (e.g. `InternalStats::CompactionStats::bytes_written`, which is
      used for calculating write amplification, now considers the blob files as well).
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7345
      
      Test Plan: Tested using `make check` and `db_bench`.
      
      Reviewed By: riversand963
      
      Differential Revision: D23506369
      
      Pulled By: ltamasi
      
      fbshipit-source-id: 646885f22dfbe063f650d38a1fedc132f499a159
      b0e78341
    • A
      Makefile support subset/individual valgrind tests (#7379) · d4993b9b
      Andrew Kryczka 提交于
      Summary:
      Introduced `valgrind_check_some`, which is analogous to the `check_some` target for non-valgrind tests. It simplifies the process for running a single valgrind test or subset of valgrind tests when trying to repro a failure.
      
      I also added a `ROCKSDBTESTS_ONLY` parameter, which simplifies selecting a single test to run. Previously the user would have to use `ROCKSDBTESTS_START` and `ROCKSDBTESTS_END`, but it was difficult to determine the end variable since it is an exclusive endpoint and must match an actual test name.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7379
      
      Reviewed By: pdillinger
      
      Differential Revision: D23673608
      
      Pulled By: ajkr
      
      fbshipit-source-id: 87ed81f1a671d46c2dff6a701f85f1891c725b3f
      d4993b9b
    • M
      Bring the Configurable options together (#5753) · 7d472acc
      mrambacher 提交于
      Summary:
      This PR merges the functionality of making the ColumnFamilyOptions, TableFactory, and DBOptions into Configurable into a single PR, resolving any merge conflicts
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5753
      
      Reviewed By: ajkr
      
      Differential Revision: D23385030
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: 8b977a7731556230b9b8c5a081b98e49ee4f160a
      7d472acc
    • A
      Add a new IOStatus subcode to indicate that writes are fenced off (#7374) · 18a3227b
      anand76 提交于
      Summary:
      In a distributed file system, directory ownership is enforced by fencing
      off the previous owner once they've been preempted by a new owner. This
      PR adds a IOStatus subcode for ```StatusCode::IOError``` to indicate this.
      Once this error is returned for a file write, the DB is put in read-only
      mode and not allowed to resume in read-write mode.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7374
      
      Test Plan: Add new unit tests in ```error_handler_fs_test```
      
      Reviewed By: riversand963
      
      Differential Revision: D23687777
      
      Pulled By: anand1976
      
      fbshipit-source-id: bef948642089dc0af399057864d9a8ca339e8b2f
      18a3227b
    • P
      Update Travis config for broken snapd on ppc (#7381) · 7e097507
      Peter Dillinger 提交于
      Summary:
      snapd update has been failing on ppc for ~a week. Disabling it
      for now in pull requests.
      
      Also, https://github.com/facebook/rocksdb/issues/6653 seems to be fixed, so re-enabling standard unit tests for
      PPC on pull requests.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7381
      
      Test Plan: CI
      
      Reviewed By: jay-zhuang
      
      Differential Revision: D23684962
      
      Pulled By: pdillinger
      
      fbshipit-source-id: 96ec9487b714c4741bb1653dae90b24118830cb5
      7e097507
    • P
      Disable sst_file_manager in stress testing backup restore (#7384) · a0ac71aa
      Peter Dillinger 提交于
      Summary:
      This is potentially the cause of failures:
      
          Failure in Destroy restore dir with: IO error: file rmdir: /dev/shm/rocksdb/rocksdb_crashtest_whitebox/.restore13: Directory not empty
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7384
      
      Test Plan: smoke test blackbox_crash_test
      
      Reviewed By: jay-zhuang
      
      Differential Revision: D23685087
      
      Pulled By: pdillinger
      
      fbshipit-source-id: 55f62e9853ce84be1d5ca7d856de867f0f2596ee
      a0ac71aa
    • T
      Implement missing Java API for ColumnFamilyOptions (#7372) · 6b72342a
      Tomasz Posluszny 提交于
      Summary:
      Covered methods:
      - OldDefaults()
      - OptimizeForSmallDb(std::shared_ptr<Cache>)
      
      Covered fields:
      - cf_paths
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7372
      
      Reviewed By: pdillinger
      
      Differential Revision: D23683449
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 3e5a8b657cc382c19de3a48c666a3b0e8d96968d
      6b72342a
  3. 13 9月, 2020 1 次提交
  4. 12 9月, 2020 1 次提交
    • Y
      Cancel tombstone skipping during bottommost compaction (#7356) · 205e5776
      Yanqin Jin 提交于
      Summary:
      During bottommost compaction, RocksDB cannot simply drop a tombstone if
      this tombstone is not in the earliest snapshot. The current behavior is: RocksDB
      skips other internal keys (of the same user key) in the same snapshot range. In
      the meantime, RocksDB should check for the `shutting_down` flag. Otherwise, it
      is possible for a bottommost compaction that has already started running to take
      a long time to finish, even if the application has tried to cancel all background jobs.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7356
      
      Test Plan: make check
      
      Reviewed By: ltamasi
      
      Differential Revision: D23663241
      
      Pulled By: riversand963
      
      fbshipit-source-id: 25f8e9b51bc3bfa3353cdf87557800f9d90ee0b5
      205e5776
  5. 11 9月, 2020 4 次提交
    • P
      Assert valid linked list for write group (#7375) · be8445ee
      Peter Dillinger 提交于
      Summary:
      We've seen some segfaults in db_write_test, with at least one
      suggesting corruption of a write group linked list. Adding an assertion
      to have this fail in a more specific way if that is the broken
      invariant.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7375
      
      Test Plan: make check
      
      Reviewed By: jay-zhuang
      
      Differential Revision: D23638477
      
      Pulled By: pdillinger
      
      fbshipit-source-id: a76fd677cad60a3a516bd363947bfd9ce418edc1
      be8445ee
    • P
      Fix cf_consistency_stress for backup/restore, harmonize (#7373) · c4e2066d
      Peter Dillinger 提交于
      Summary:
      We can only check key on restored backup if in a stress test
      configuration locking the key. (Fixes mismatch seen in backup/restore
      with atomic flush.)
      
      TestCheckpoint used a very ugly solution to the same problem: copy-paste
      dozens of lines of code with some changes and removals. I removed the
      unnecessary implementation and made the existing one simply adaptive,
      like TestBackupRestore.
      
      Also made TestBackupRestore clean up dead backup/restore directories on
      success.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7373
      
      Test Plan:
      blackbox_crash_test_with_atomic_flush for a while,
      blackbox_crash_test for a while, with backup and checkpoint 1 in 5k and
      only 1k max_keys to stress this area
      
      Reviewed By: ajkr
      
      Differential Revision: D23629057
      
      Pulled By: pdillinger
      
      fbshipit-source-id: d7fe7e2be75aaf3cf974be9540a7c5c5de8b371b
      c4e2066d
    • P
      Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) · 92639b93
      Peter Dillinger 提交于
      Summary:
      https://github.com/facebook/rocksdb/issues/3341 guaranteed that upon return of `GetSortedWalFiles` after
      `DisableFileDeletions`, all pending purges of previously obsolete WAL
      files will have finished. However, the addition of
      avoid_unnecessary_blocking_io in https://github.com/facebook/rocksdb/issues/5043 opened a hole in the code making
      that assurance, which can lead to files to be copied for checkpoint or
      backup going missing before being copied, with that option enabled.
      
      This change patches the hole.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7369
      
      Test Plan:
      apparent fix to backups in crash test observed. Will work
      on a unit test for another commit
      
      Reviewed By: ajkr
      
      Differential Revision: D23620258
      
      Pulled By: pdillinger
      
      fbshipit-source-id: bea36b461a5b719c3e3ef802f967bc3e8ae71614
      92639b93
    • L
      Expose the start of the expiration range for TTL blob files through LiveFileMetaData (#7365) · 5ce246c7
      Levi Tamasi 提交于
      Summary:
      The patch adds support for exposing the start of the expiration range
      for TTL blob files through the `GetLiveFilesMetaData` API. This can be
      used for monitoring purposes, i.e. to make sure TTL blob files are
      deleted in a timely manner. The patch also fixes a couple of uninitialized
      variable issues.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7365
      
      Test Plan: `make check`
      
      Reviewed By: pdillinger
      
      Differential Revision: D23605465
      
      Pulled By: ltamasi
      
      fbshipit-source-id: 97a9612bf5f4b058423debdd3f28f576bb23a70f
      5ce246c7
  6. 10 9月, 2020 7 次提交
  7. 09 9月, 2020 7 次提交
    • P
      Fix some errors showing up in Travis builds (#7359) · 9de912de
      Peter Dillinger 提交于
      Summary:
      Also enables a pull request to trigger all the Travis
      configurations by writing FULL_CI in the commit message. (See what I did
      there?)
      
      First issue
      
          make: *** No rule to make target 'jl/util/crc32c_ppc_asm.o', needed by 'rocksdbjava'.  Stop.
      
      Second issue
      
          tools/db_bench_tool.cc:5514:38: error: ‘gen_exp.rocksdb::Benchmark::GenerateTwoTermExpKeys::keyrange_size_’ may be used uninitialized in this function
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7359
      
      Test Plan: CI
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D23582132
      
      Pulled By: pdillinger
      
      fbshipit-source-id: 06d794673fd522ba11cf6398385387e6bd97ef89
      9de912de
    • A
      Use FSRandomRWFilePtr Object to call underlying file system. (#7198) · 0de335e0
      Akanksha Mahajan 提交于
      Summary:
      Replace FSRandomRWFile pointer with FSRandomRWFilePtr object in the rocksdb internal code.
      This new object wraps FSRandomRWFile pointer.
      
      Objective: If tracing is enabled, FSRandomRWFile object returns FSRandomRWFileTracingWrapper pointer that includes all necessary information in IORecord and calls underlying FileSystem and invokes IOTracer to dump that record in a binary file. If tracing is disabled then, underlying FileSystem pointer is returned directly.
      FSRandomRWFilePtr wrapper class is added to bypass the FSRandomRWFileWrapper when
      tracing is disabled.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7198
      
      Test Plan: make check -j64
      
      Reviewed By: anand1976
      
      Differential Revision: D23421116
      
      Pulled By: akankshamahajan15
      
      fbshipit-source-id: 8a5ba0e7d9c1ba34c3a6f29829b107c5f09ab6a3
      0de335e0
    • J
      Fix compile error for old gcc-4.8 (#7358) · 8a8a01c6
      Jay Zhuang 提交于
      Summary:
      gcc-4.8 returns error when using the constructor. Not sure if it's a compiler bug/limitation or code issue:
      ```
      table/block_based/block_based_table_reader.cc:3183:67: error: use of deleted function ‘rocksdb::WritableFileStringStreamAdapter::WritableFileStringStreamAdapter(rocksdb::WritableFileStringStreamAdapter&&)’
      ```
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7358
      
      Reviewed By: pdillinger
      
      Differential Revision: D23577651
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: b0197e3d3538da61a6f3866410d88d2047fb9695
      8a8a01c6
    • Y
      Update HISTORY.md for PR7329 (#7355) · 8307d440
      Yanqin Jin 提交于
      Summary:
      As title.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7355
      
      Reviewed By: pdillinger
      
      Differential Revision: D23566635
      
      Pulled By: riversand963
      
      fbshipit-source-id: f8d846bcff637e7617b764b7bfb9a948ea18d195
      8307d440
    • A
      Store FSWritableFilePtr object in WritableFileWriter (#7193) · b175eceb
      Akanksha Mahajan 提交于
      Summary:
      Replace FSWritableFile pointer with FSWritableFilePtr
          object in WritableFileWriter.
          This new object wraps FSWritableFile pointer.
      
          Objective: If tracing is enabled, FSWritableFile Ptr returns
          FSWritableFileTracingWrapper pointer that includes all necessary
          information in IORecord and calls underlying FileSystem and invokes
          IOTracer to dump that record in a binary file. If tracing is disabled
          then, underlying FileSystem pointer is returned directly.
          FSWritableFilePtr wrapper class is added to bypass the
          FSWritableFileWrapper when
          tracing is disabled.
      
          Test Plan: make check -j64
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7193
      
      Reviewed By: anand1976
      
      Differential Revision: D23355915
      
      Pulled By: akankshamahajan15
      
      fbshipit-source-id: e62a27a13c1fd77e36a6dbafc7006d969bed25cf
      b175eceb
    • P
      Fix backup/restore in stress/crash test (#7357) · 4e258d3e
      Peter Dillinger 提交于
      Summary:
      (1) Skip check on specific key if restoring an old backup
      (small minority of cases) because it can fail in those cases. (2) Remove
      an old assertion about number of column families and number of keys
      passed in, which is broken by atomic flush (cf_consistency) test. Like
      other code (for better or worse) assume a single key and iterate over
      column families. (3) Apply mock_direct_io to NewSequentialFile so that
      db_stress backup works on /dev/shm.
      
      Also add more context to output in case of backup/restore db_stress
      failure.
      
      Also a minor fix to BackupEngine to report first failure status in
      creating new backup, and drop another clue about the potential
      source of a "Backup failed" status.
      
      Reverts "Disable backup/restore stress test (https://github.com/facebook/rocksdb/issues/7350)"
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7357
      
      Test Plan:
      Using backup_one_in=10000,
      "USE_CLANG=1 make crash_test_with_atomic_flush" for 30+ minutes
      "USE_CLANG=1 make blackbox_crash_test" for 30+ minutes
      And with use_direct_reads with TEST_TMPDIR=/dev/shm/rocksdb
      
      Reviewed By: riversand963
      
      Differential Revision: D23567244
      
      Pulled By: pdillinger
      
      fbshipit-source-id: e77171c2e8394d173917e36898c02dead1c40b77
      4e258d3e
    • L
      Clean up SubcompactionState a bit (#7322) · 423d0511
      Levi Tamasi 提交于
      Summary:
      The patch cleans up a few things in `CompactionJob::SubcompactionState`:
      
      * Instead of using both the member initializer list and in-class initializers (and
      sometimes both at the same time for the same member), the struct now uniformly
      uses the latter to initialize integer members.
      * The default parameter value for the constructor parameter `size` is removed.
      * The explicitly deleted copy operations are removed, since they are implicitly deleted
      anyways because of the `unique_ptr` members.
      * The handwritten move operations, which did not move the member `c_iter` and
      were not declared `nothrow`, are removed. Note that with the user-declared copy
      operations gone (see the previous item), we can rely on the compiler to (correctly)
      generate these methods.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7322
      
      Test Plan: `make check`
      
      Reviewed By: siying
      
      Differential Revision: D23382408
      
      Pulled By: ltamasi
      
      fbshipit-source-id: a4ae5af150161c50ff7bdc07fa145482d0150bfe
      423d0511
  8. 05 9月, 2020 3 次提交
    • Y
      Add a new stats level to exclude tickers (#7329) · ab202e8d
      Yanqin Jin 提交于
      Summary:
      Currently, application may pass a statistics object to db but later
      wants to reduce stats tracking overhead by setting stats level to
      kExceptHistogramOrTimers (the current lowest level). Tickers will still
      be incremented, causing up to 1% CPU. We can add a new lowest stats
      level `kExceptTickers` to disable ticker incrementing as well, thus
      reducing CPU cycles spent on tickers.
      
      Test Plan (devserver):
      ```
      make check
      make clean
      DEBUG_LEVEL=0 make db_bench
      ./db_bench -perf_level=1 -stats_level=0 -statistics -benchmarks=fillseq,readrandom -duration=120
      ```
      
      Measure CPU util (%) before and after change:
      CPU util by rocksdb::RecordTick: 1.1 vs (<0.1)
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7329
      
      Reviewed By: pdillinger
      
      Differential Revision: D23434014
      
      Pulled By: riversand963
      
      fbshipit-source-id: 72ff0f02a192ac476d4b0044b9f37fd4a22ff0d4
      ab202e8d
    • J
      Add sst_file_dumper status check (#7315) · 27aa443a
      Jay Zhuang 提交于
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7315
      
      Test Plan:
      `ASSERT_STATUS_CHECKED=1 make sst_dump_test && ./sst_dump_test`
      And manually run `./sst_dump --file=*.sst` before and after the change.
      
      Reviewed By: pdillinger
      
      Differential Revision: D23361669
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 5bf51a2a90ee35c8c679e5f604732ec2aef5949a
      27aa443a
    • J
      Disable backup/restore stress test (#7350) · ef32f110
      Jay Zhuang 提交于
      Summary:
      Seems it's causing some tests failures.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7350
      
      Reviewed By: riversand963
      
      Differential Revision: D23544109
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 798a0ca374a20b6c2d0f29582729ff101c6a2e99
      ef32f110
  9. 04 9月, 2020 9 次提交
    • P
      Add file checksum to stress/crash test (#7343) · 06ad5dd2
      Peter Dillinger 提交于
      Summary:
      This change has the crash test randomly select from a few file
      checksum implementations, or nullptr, for DB file_checksum_gen_factory.
      For compatibility across runs on same DB, each non-null factory can
      understand all the other functions, but the default changes.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7343
      
      Test Plan:
      'make blackbox_crash_test' for a while, including with some
      debug output to ensure code is being exercised.
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D23494580
      
      Pulled By: pdillinger
      
      fbshipit-source-id: 73bbc7ca32c1adaf619134c0c830f12894880b8a
      06ad5dd2
    • C
      Fix wrong level args (#7346) · 3f9b7560
      Cheng Chang 提交于
      Summary:
      The level args should be output level instead of input levels.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7346
      
      Test Plan: make check
      
      Reviewed By: ajkr
      
      Differential Revision: D23506373
      
      Pulled By: cheng-chang
      
      fbshipit-source-id: b2f701d44c13581c5c10c4dbebded4fcd354d641
      3f9b7560
    • P
      Fix, enable, and enhance backup/restore in db_stress (#7348) · 499c9448
      Peter Dillinger 提交于
      Summary:
      Although added to db_stress, testing of backup/restore
      was never integrated into the crash test, originally concerned about
      performance. I've enabled it now and to address the peformance concern,
      testing backup/restore is always skipped once the db exceeds a certain
      size threshold, default 100MB. This should provide sufficient
      opportunity for testing BackupEngine without bogging down everything
      else with heavier and heavier operations.
      
      Also fixed backup/restore in db_stress by making sure PurgeOldBackups
      can remove manifest files, which are normally kept around for db_stress.
      
      Added more coverage of backup options, and up to three backups being
      saved in one backup directory (in some cases).
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7348
      
      Test Plan:
      ran 'make blackbox_crash_test' for a while, with heightened
      probabilitly of taking backups (1/10k). Also confirmed with some debug
      output that the code is being covered, TestBackupRestore only takes
      a few seconds to complete when triggered, and even at 1/10k and ~50MB
      database, there's <,~ 1 thread testing backups at any time.
      
      Reviewed By: ajkr
      
      Differential Revision: D23510835
      
      Pulled By: pdillinger
      
      fbshipit-source-id: b6b8735591808141f81f10773ac31634cf03b6c0
      499c9448
    • A
      add `ldb unsafe_remove_sst_file` subcommand (#7335) · 57467673
      Andrew Kryczka 提交于
      Summary:
      This is adapted from https://github.com/facebook/rocksdb/issues/6678 but takes a different approach, avoiding opening a read-write DB and avoiding the `DeleteFile()` API.
      
      First, this PR refactors how options variables are initialized in `ldb` so it can be reused in a subcommand that doesn't open a DB:
      
      - Separated remaining option initialization logic out of `OpenDB()`. The new `PrepareOptions()` function initializes the full options state.
      - Fixed an old TODO about applying the subcommand CF option overrides to the proper `ColumnFamilyOptions` object.
      
      Second, this PR adds the `ldb unsafe_remove_sst_file` subcommand. It uses the `VersionSet`-level APIs to remove the file with the specified number.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7335
      
      Test Plan: played with interactive python and this file removal command. Verified openability/correct results in case of multiple column families, multiple levels, etc.
      
      Reviewed By: pdillinger
      
      Differential Revision: D23454575
      
      Pulled By: ajkr
      
      fbshipit-source-id: 039b7a8cbfc42fd123dcb25821eef51d61148afe
      57467673
    • A
      add warning on `DeleteFile()` API (#7337) · 40e97b02
      Andrew Kryczka 提交于
      Summary:
      Since we can't land https://github.com/facebook/rocksdb/issues/7336 until the next major release, added a strong warning against the `DeleteFile()` API in the meantime.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7337
      
      Reviewed By: pdillinger
      
      Differential Revision: D23459728
      
      Pulled By: ajkr
      
      fbshipit-source-id: 326cb9b18190386080c35c761a8736d8a877dafb
      40e97b02
    • A
      fix SstFileWriter with dictionary compression (#7323) · af54c409
      Andrew Kryczka 提交于
      Summary:
      In block-based table builder, the cut-over from buffered to unbuffered
      mode involves sampling the buffered blocks and generating a dictionary.
      There was a bug where `SstFileWriter` passed zero as the `target_file_size`
      causing the cutover to happen immediately, so there were no samples
      available for generating the dictionary.
      
      This PR changes the meaning of `target_file_size == 0` to mean buffer
      the whole file before cutting over. It also adds dictionary compression
      support to `sst_dump --command=recompress` for easy evaluation.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7323
      
      Reviewed By: cheng-chang
      
      Differential Revision: D23412158
      
      Pulled By: ajkr
      
      fbshipit-source-id: 3b232050e70ef3c2ee85a4b5f6fadb139c569873
      af54c409
    • E
      Expose rocksdb_open_column_families_with_ttl C function (#7314) · 5b1ccdc1
      Eduardo Barreto Alexandre 提交于
      Summary:
      This PR creates `rocksdb_open_column_families_with_ttl` which allows C API users to open a DBWithTLL with column families.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7314
      
      Reviewed By: cheng-chang
      
      Differential Revision: D23430287
      
      Pulled By: ajkr
      
      fbshipit-source-id: 307aa21d170d1402653263a91f6f832ef76afba0
      5b1ccdc1
    • H
      Avoid converting MERGES to PUTS when allow_ingest_behind is true (#7166) · d0c1a01c
      Hiep 提交于
      Summary:
      - Closes https://github.com/facebook/rocksdb/issues/6490
      - Currently MERGEs are converted to PUTs at bottom or compaction has reached the beginning of the key, this can wrongly cover a PUT future base case.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7166
      
      Test Plan:
      - Automated: `make all check`
      - Manual: With `allow_ingest_behind = true`, add Merge operations to a key then run compaction. Then run ingesting external files to make sure the base case is probably compacted with existing Merges.
      
      Reviewed By: cheng-chang
      
      Differential Revision: D23325425
      
      Pulled By: ajkr
      
      fbshipit-source-id: 3eb415eb7b381b5453e45245393566153b1abb68
      d0c1a01c
    • H
      Close databases on benchmark error exits in db_bench (#7327) · 679a413f
      Hans Holmberg 提交于
      Summary:
      Delete database instances to make sure there are no loose threads
      running before exit(). This fixes segfaults seen when running
      workloads through CompositeEnvs with custom file systems.
      
      For further background on the issues arising when using CompositeEnvs, see the discussion in:
      https://github.com/facebook/rocksdb/pull/6878
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7327
      
      Reviewed By: cheng-chang
      
      Differential Revision: D23433244
      
      Pulled By: ajkr
      
      fbshipit-source-id: 4e19cf2067e3fe68c2a3fe1823f24b4091336bbe
      679a413f