1. 22 7月, 2020 2 次提交
    • H
      Use steady_clock instead of system_clock in FileOperationInfo::TimePoint (#7153) · dbc51adb
      Haosen Wen 提交于
      Summary:
      Issue https://github.com/facebook/rocksdb/issues/7133 reported that using `system_clock` in `FileOperationInfo::TimePoint` causes the duration of file flush operation (which can be a noop on MacOS in some scenarios) appears to be 0 and fail an assertion in listener_test. Using `steady_clock` supposedly fixed the problem.
      `steady_clock` actually fits better into the use cases of `FileOperationInfo::TimePoint` as all usages care about durations but not wall clock time.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7153
      
      Test Plan: make check.
      
      Reviewed By: riversand963
      
      Differential Revision: D22654136
      
      Pulled By: roghnin
      
      fbshipit-source-id: 5980b1080734bdae496a18071a2c2b5887c67d85
      dbc51adb
    • Z
      BackupEngine computes table checksums only once if db session ids are available (#7110) · b923dc72
      Zitan Chen 提交于
      Summary:
      BackupEngine requires computing table checksums twice when backing up table files to the `shared_checksum` directory.
      
      The repeated computation can be avoided by utilizing the db session id stored as a part of the table properties.
      
      Filenames of table files in the `shared_checksum` directory depend on the following conditions:
      1. the naming scheme is `kOptionalChecksumAndDbSessionId`,
      2. `db_session_id` is not empty,
      3. checksum is available in the DB manifest.
      
      If 1,2,3 are satisfied, then the filenames will be of the form `<file_number>_<checksum>_<db_session_id>.sst`.
      If 1,2 are satisfied, then the filenames will be of the form `<file_number>_<db_session_id>.sst`.
      In all other cases, the filenames are of the form `<file_number>_<checksum>_<size>.sst`.
      
      Additionally, if `kOptionalChecksumAndDbSessionId` is used (and not falling back to `kChecksumAndFileSize`), the `<checksum>` appeared in the filenames is hexadecimally encoded, instead of being plain `uint32_t` value.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7110
      
      Test Plan: backupable_db_test and manual tests.
      
      Reviewed By: ajkr
      
      Differential Revision: D22508992
      
      Pulled By: gg814
      
      fbshipit-source-id: 5669f0ea9ad5a097f69f6d87aca4abba15032389
      b923dc72
  2. 21 7月, 2020 5 次提交
    • S
      Exclude two tests in CircleCI TSAN tests (#7152) · 0f487cc3
      sdong 提交于
      Summary:
      Two TSAN tests occaionaly fail. Exclude them for now:
      
      [ RUN      ] DeleteFileTest.BackgroundPurgeCFDropTest
      db/deletefile_test.cc:122: Failure
      Expected equality of these values:
        required_manifest
          Which is: 1
        manifest_cnt
          Which is: 2
      
      [ RUN      ] FormatLatest/ColumnFamilyTest.FlushCloseWALFiles/0
      db/column_family_test.cc:3004: Failure
      Expected equality of these values:
        2
        env.num_open_wal_file_.load()
          Which is: 1
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7152
      
      Test Plan: Watch CircleCI restuls
      
      Reviewed By: ajkr
      
      Differential Revision: D22632285
      
      fbshipit-source-id: 29fa348e8be917be0237c74812a8b0b04978e84e
      0f487cc3
    • S
      column_family_test: fix a data race related to sleeping task (#7150) · 1cf4731d
      sdong 提交于
      Summary:
      TSAN reports warning in one column_family_test:
      
      WARNING: ThreadSanitizer: data race (pid=16352)
        Write of size 8 at 0x7ffcdf042158 by main thread:
          #0 pthread_cond_destroy <null> (column_family_test+0x471f65)
          https://github.com/facebook/rocksdb/issues/1 rocksdb::port::CondVar::~CondVar() /home/circleci/project/port/port_posix.cc:101:49 (column_family_test+0x8a627a)
          https://github.com/facebook/rocksdb/issues/2 rocksdb::test::SleepingBackgroundTask::~SleepingBackgroundTask() /home/circleci/project/./test_util/testutil.h:397:7 (column_family_test+0x54b6e2)
          https://github.com/facebook/rocksdb/issues/3 rocksdb::ColumnFamilyTest_FlushCloseWALFiles_Test::TestBody() /home/circleci/project/db/column_family_test.cc:3008:1 (column_family_test+0x54b6e2)
      ......
        Previous read of size 8 at 0x7ffcdf042158 by thread T2 (mutexes: write M0):
          #0 pthread_cond_broadcast <null> (column_family_test+0x471dd2)
          https://github.com/facebook/rocksdb/issues/1 rocksdb::port::CondVar::SignalAll() /home/circleci/project/port/port_posix.cc:139:28 (column_family_test+0x8a651a)
          https://github.com/facebook/rocksdb/issues/2 rocksdb::test::SleepingBackgroundTask::DoSleep() /home/circleci/project/./test_util/testutil.h:412:12 (column_family_test+0x58574b)
      ......
      
      Likely, SleepingBackgroundTask::DoSleep() started to execute after the main thread has finished everything, cancelled and waited for sleeping tasks to finish. At this time, although DoSlee() will not sleep, but it also accesses the mutex, creating a data race with destructor of the test. Fix this bug by waiting for the sleeping task to start sleeping after it is scheduled.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7150
      
      Test Plan: Run these modified tests and make sure it doesn't break.
      
      Reviewed By: riversand963
      
      Differential Revision: D22630716
      
      fbshipit-source-id: cc5781cf69083685de406490438898238bdfc2d3
      1cf4731d
    • A
      minimize BlockIter comparator scope (#7149) · 643c863b
      Andrew Kryczka 提交于
      Summary:
      PR https://github.com/facebook/rocksdb/issues/6944 transitioned `BlockIter` from using `Comparator*` to using
      concrete `UserComparatorWrapper` and `InternalKeyComparator`. However,
      adding them as instance variables to `BlockIter` was not optimal.
      Bloating `BlockIter` caused the `ArenaWrappedDBIter`'s arena allocator to do more heap
      allocations (in certain cases) which harmed performance of `DB::NewIterator()`. This PR
      pushes down the concrete comparator objects to the point of usage, which
      forces them to be on the stack. As a result, the `BlockIter` is back to
      its original size prior to https://github.com/facebook/rocksdb/issues/6944 (actually a bit smaller since there
      were two `Comparator*` before).
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7149
      
      Test Plan:
      verified our internal `DB::NewIterator()`-heavy regression
      test no longer reports regression.
      
      Reviewed By: riversand963
      
      Differential Revision: D22623189
      
      Pulled By: ajkr
      
      fbshipit-source-id: f6d69accfe5de51e0bd9874a480b32b29909bab6
      643c863b
    • S
      Fix a minor data race in stats dumping threads initialization (#7151) · 98707044
      sdong 提交于
      Summary:
      https://github.com/facebook/rocksdb/pull/7145 creates a minor data race against the stat creation counter. Turn it to atomic.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7151
      
      Test Plan: Run the test.
      
      Reviewed By: ajkr
      
      Differential Revision: D22631014
      
      fbshipit-source-id: c6fb69ac5b9df7139795dacea5ce9fb9fd3278d7
      98707044
    • J
      Store the test results to CircleCI (#7137) · 77062cf1
      Jay Zhuang 提交于
      Summary:
      To have test report.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7137
      
      Reviewed By: siying
      
      Differential Revision: D22630798
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: bc07ba673c0bceed5a4829b4af2d9a74435379c7
      77062cf1
  3. 18 7月, 2020 5 次提交
  4. 17 7月, 2020 1 次提交
    • M
      Add Support for saving CompressionOptions to Options File (#6817) · ec711b23
      mrambacher 提交于
      Summary:
      This PR does a few things:
      - The "compression_opts" and "bottom_compression_opts" can now be read/written as name/value pairs of options (instead of only a colon-separated list;
      - These options can now be read/written to the Options file;
      - The parallel_threads value can now be set (either in the colon or name-value format).
      
      The compression options are now stored and treated as a OptionTypeInfo::Struct by the options system, meaning they can be read and written like the other structs.  This change allows them to be read/written easily to the options file.
      
      Additionally, the colon-format was extended to allow support for setting parallel threads.  Tests were added to test all of the option settings via the optional parameters in the colon format.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6817
      
      Reviewed By: ajkr
      
      Differential Revision: D22396004
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: 38bcf74b7e9cd5bc2a84540fac2e9ba4f765b2c8
      ec711b23
  5. 16 7月, 2020 7 次提交
    • L
      Remove some more dead code around syncing blob files (#7138) · c5ddeceb
      Levi Tamasi 提交于
      Summary:
      Periodic syncing of blob files is handled by a lower layer, namely by
      `WritableFileWriter`; the `NeedsFsync` method of `BlobFile` and the
      `last_fsync_` member variable are actually unused and thus can be
      removed. See also https://github.com/facebook/rocksdb/pull/7125 .
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7138
      
      Test Plan: `make check`
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D22562981
      
      Pulled By: ltamasi
      
      fbshipit-source-id: c235aad94a7c27120528c9ec270a7a5b9154e49f
      c5ddeceb
    • A
      Add "build-examples" in CircleCI (#7136) · a7feebd6
      Akanksha Mahajan 提交于
      Summary:
      Add "examples" build (which build examples folder in rocksdb) in TravisCI to CircleCI. This is helpful before pull request.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7136
      
      Test Plan: Watch for CircleCI results to succeed
      
      Reviewed By: jay-zhuang
      
      Differential Revision: D22555528
      
      Pulled By: akankshamahajan15
      
      fbshipit-source-id: 6bca16647760d5f0131f064765fe9e88e034c578
      a7feebd6
    • S
      Suppress a TSAN warning (#7126) · ca5a069a
      sdong 提交于
      Summary:
      TSAN shows warning with clang with warning similar to this:
      
      WARNING: ThreadSanitizer: data race (pid=10159)
        Atomic write of size 8 at 0x7b5000002890 by thread T33:
          #0 __tsan_atomic64_store <null> (db_test+0x4ca2b5)
          https://github.com/facebook/rocksdb/issues/1 std::__atomic_base<unsigned long>::store(unsigned long, std::memory_order) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/atomic_base.h:374:2 (db_test+0x774fde)
          https://github.com/facebook/rocksdb/issues/2 rocksdb::VersionSet::SetLastSequence(unsigned long) /home/circleci/project/./db/version_set.h:1057:20 (db_test+0x774fde)
          https://github.com/facebook/rocksdb/issues/3 rocksdb::DBImpl::WriteImpl(rocksdb::WriteOptions const&, rocksdb::WriteBatch*, rocksdb::WriteCallback*, unsigned long*, unsigned long, bool, unsigned long*, unsigned long, rocksdb::PreReleaseCallback*) /home/circleci/project/db/db_impl/db_impl_write.cc:449:18 (db_test+0x774fde)
      ......
        Previous read of size 8 at 0x7b5000002890 by thread T5 (mutexes: write M1044689462619020832):
          #0 rocksdb::DBImpl::ReleaseSnapshot(rocksdb::Snapshot const*) /home/circleci/project/db/db_impl/db_impl.cc (db_test+0x6f4ae7)
          https://github.com/facebook/rocksdb/issues/1 rocksdb::(anonymous namespace)::MTThreadBody(void*) /home/circleci/project/db/db_test.cc:2514:13 (db_test+0x56ac59)
          https://github.com/facebook/rocksdb/issues/2 rocksdb::(anonymous namespace)::StartThreadWrapper(void*) /home/circleci/project/env/env_posix.cc:443:3 (db_test+0x88c4cd)
      
      It is not limited to ReleaseSnapshot() and rocksdb::DBImpl::MultiCFSnapshot().
      
      While we are not 100% sure it doesn't indicate any correctness violation, we suppress them for now to keep TSAN clean with more tests so that we can cover more bugs with CI.
      
      In the gcc runs we have been running, this warning rarely shows up.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7126
      
      Test Plan: See the mini-TSAN test to pass with reasonable run time.
      
      Reviewed By: ajkr
      
      Differential Revision: D22552375
      
      fbshipit-source-id: ebdd3854cb3becec3403970326a1ca961db2ab00
      ca5a069a
    • L
      Turn the compression_type check in BlobDBImpl::DecompressSlice into an assertion (#7127) · ee8c79d4
      Levi Tamasi 提交于
      Summary:
      In both cases where `BlobDBImpl::DecompressSlice` is called,
      `compression_type` is already checked at the call site; thus, the check
      inside the method is redundant and can be turned into an assertion.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7127
      
      Test Plan: `make check`
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D22533454
      
      Pulled By: ltamasi
      
      fbshipit-source-id: ae524443fc6abe0a5fb12327a3fe761a9cd2c831
      ee8c79d4
    • J
      CircleCI: Pipe java-build to ignore EAGAIN errors (#7135) · afb6bb1d
      Jay Zhuang 提交于
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7135
      
      Reviewed By: siying
      
      Differential Revision: D22553594
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 5b40621ffad05c4fffe3ac536a11d6c87e7ef512
      afb6bb1d
    • Z
      Auto resume the DB from Retryable IO Error (#6765) · a10f12ed
      Zhichao Cao 提交于
      Summary:
      In current codebase, in write path, if Retryable IO Error happens, SetBGError is called. The retryable IO Error is converted to hard error and DB is in read only mode. User or application needs to resume it. In this PR, if Retryable IO Error happens in one DB, SetBGError will create a new thread to call Resume (auto resume). otpions.max_bgerror_resume_count controls if auto resume is enabled or not (if max_bgerror_resume_count<=0, auto resume will not be enabled). options.bgerror_resume_retry_interval controls the time interval to call Resume again if the previous resume fails due to the Retryable IO Error. If non-retryable error happens during resume, auto resume will terminate.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/6765
      
      Test Plan: Added the unit test cases in error_handler_fs_test and pass make asan_check
      
      Reviewed By: anand1976
      
      Differential Revision: D21916789
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: acb8b5e5dc3167adfa9425a5b7fc104f6b95cb0b
      a10f12ed
    • Y
      Fix typo in ThreadData comment (#7131) · e8d5a248
      yxj25245 提交于
      Summary:
      Fix typo in ThreadData comment
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7131
      
      Reviewed By: riversand963
      
      Differential Revision: D22543135
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 39c9d0e8cd5a364af9a2f05fd3783e8482dea976
      e8d5a248
  6. 15 7月, 2020 6 次提交
    • Y
      Report corrupted keys during compaction (#7124) · 27735dea
      Yanqin Jin 提交于
      Summary:
      Currently, RocksDB lets compaction to go through even in case of
      corrupted keys, the number of which is reported in CompactionJobStats.
      However, RocksDB does not check this value. We should let compaction run
      in a stricter mode.
      
      Temporarily disable two tests that allow corrupted keys in compaction.
      With this PR, the two tests will assert(false) and terminate. Still need
      to investigate what is the recommended google-test way of doing it.
      Death test (EXPECT_DEATH) in gtest has warnings now.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7124
      
      Test Plan: make check
      
      Reviewed By: ajkr
      
      Differential Revision: D22530722
      
      Pulled By: riversand963
      
      fbshipit-source-id: 6a5a6a992028c6d4f92cb74693c92db462ae4ad6
      27735dea
    • L
      Update some log messages in BlobDB to account for compaction filters (#7128) · 687fbd02
      Levi Tamasi 提交于
      Summary:
      https://github.com/facebook/rocksdb/pull/6850, which added compaction
      filter support to BlobDB, reused elements of the BlobDB GC mechanism.
      This patch updates some log messages in this logic to account for this
      fact; namely, it replaces mentions of "GC" with "compaction/GC" to avoid
      confusion in cases when GC is not enabled.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7128
      
      Test Plan: `make check`
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D22535371
      
      Pulled By: ltamasi
      
      fbshipit-source-id: 1f14f3b02ab9983728bbca1cf680420208d9a195
      687fbd02
    • A
      Only check for python location once (#7123) · de8c92a5
      Adam Retter 提交于
      Summary:
      This fixes an issue introduced in 0c56fc4d whereby the location of Python is evaluated many times and leads to excessive logging of unknown python locations of CentOS 6.
      
      The location is now only checked once.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7123
      
      Reviewed By: zhichao-cao
      
      Differential Revision: D22532274
      
      Pulled By: ajkr
      
      fbshipit-source-id: cade71b4b46e9a23d63ecb4dd36a4ac8ae217970
      de8c92a5
    • S
      Run a subset of TSAN tests in CIrcleCI (#7122) · e930ba1d
      sdong 提交于
      Summary:
      It is helpful to add some TSAN coverage before a pull request is committed. This diff adds some of them.
      Some slow tests are excluded for the running speed. Some are blacklisted because they show warnings. Will investigate these warnings and see whether we can fix or suppress them.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7122
      
      Test Plan: Watch CIrcleCI runs
      
      Reviewed By: riversand963
      
      Differential Revision: D22532133
      
      fbshipit-source-id: 81ddd02d9df19c513a12811979e8ddabae911354
      e930ba1d
    • L
      Remove some dead code from BlobLogWriter (#7125) · bdf4de6c
      Levi Tamasi 提交于
      Summary:
      Periodic syncing of blob files is performed by `WritableFileWriter`;
      `bytes_per_sync_` and `next_sync_offset_` in `BlobLogWriter` are
      actually unused (or more precisely, only used by methods that are
      themselves unused). The patch removes all this dead code.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7125
      
      Test Plan: `make check`
      
      Reviewed By: riversand963
      
      Differential Revision: D22531021
      
      Pulled By: ltamasi
      
      fbshipit-source-id: 6b293ad5a79d3e6bf15c5c68f7aedd7ce7a15f10
      bdf4de6c
    • J
      Add stress test for GetProperty (#7111) · fc4d5f50
      Jay Zhuang 提交于
      Summary:
      Add stress test coverage for `DB::GetProperty()`.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7111
      
      Test Plan:
      ```
      ./db_stress -get_property_one_in=1
      make crash_test
      ```
      
      Reviewed By: ajkr
      
      Differential Revision: D22487906
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: c118d95cc9b4e2fa669a06e6aa531541fa885dc5
      fc4d5f50
  7. 14 7月, 2020 5 次提交
    • Y
      Report corruption on unrecognized value type (#7121) · c628fae6
      Yanqin Jin 提交于
      Summary:
      During memtable lookup, an unrecognized value type should be reported as
      Status::Corruption.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7121
      
      Test Plan: make check
      
      Reviewed By: cheng-chang
      
      Differential Revision: D22512124
      
      Pulled By: riversand963
      
      fbshipit-source-id: 9b97be7d9b230c5aae9205f96054420e5ea09066
      c628fae6
    • S
      Add VS2017 to CircleCI Runs (#7120) · 2f32d504
      sdong 提交于
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7120
      
      Test Plan: Watch CI results.
      
      Reviewed By: cheng-chang
      
      Differential Revision: D22516995
      
      fbshipit-source-id: ddf7c0482dfac30d9044b833852348eda987cb67
      2f32d504
    • A
      Add FileSystem wrapper classes for IO tracing. (#7002) · d93bd3ce
      Akanksha Mahajan 提交于
      Summary:
      1. Add the wrapper classes FileSystemTracingWrapper, FSSequentialFileTracingWrapper, FSRandomAccessFileTracingWrapper, FSWritableFileTracingWrapper, FSRandomRWFileTracingWrapper that forward the calls to underlying storage system and then pass the file operation information to IOTracer. IOTracer dumps the record in binary format for tracing.
      2. Add the wrapper classes FileSystemPtr, FSSequentialFilePtr, FSRandomAccessFilePtr, FSWritableFilePtr and FSRandomRWFilePtr that overload operator-> and return ptr to underlying storage system or Tracing wrapper class based on enabling/disabling of IO tracing. These classes are added to bypass Tracing Wrapper classes when we disable tracing.
      3. Add enums in trace.h that distinguish which options need to be added for different file operations(Read, close, write etc) as part of tracing record.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7002
      
      Test Plan: make check -j64
      
      Reviewed By: anand1976
      
      Differential Revision: D22127897
      
      Pulled By: akankshamahajan15
      
      fbshipit-source-id: 74cff58ce5661c9a3832dfaa52483f3b2d8565e0
      d93bd3ce
    • J
      Add circleci java build (#7119) · 0ff752cf
      Jay Zhuang 提交于
      Summary:
      Add circleci java build workflow.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7119
      
      Reviewed By: siying
      
      Differential Revision: D22512426
      
      Pulled By: jay-zhuang
      
      fbshipit-source-id: 45a7445c861fee48017bae42cdb7172c8b091475
      0ff752cf
    • S
      Add CLANG analyze to CircleCI (#7114) · 43cc622d
      sdong 提交于
      Summary:
      CLANG analyze is useful before pull request. Add it.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7114
      
      Test Plan: Watch the CI results to succeed.
      
      Reviewed By: riversand963
      
      Differential Revision: D22491942
      
      fbshipit-source-id: 9ccad91c6142fedc3d3dd491cf55054827908f36
      43cc622d
  8. 11 7月, 2020 5 次提交
  9. 10 7月, 2020 4 次提交
    • Z
      Fix status message size assert (#7045) · a9a97386
      Zhichao Cao 提交于
      Summary:
      In status.cc, the assert is `assert(sizeof(msgs) > index)`; msgs is a const char* array, sizeof(msgs) is the array size*char* size, which will make the assert pass all the time. Change it to sizeof(msgs)/sizeof(char*) > index.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7045
      
      Test Plan: pass make check
      
      Reviewed By: cheng-chang
      
      Differential Revision: D22291337
      
      Pulled By: zhichao-cao
      
      fbshipit-source-id: 4ba8ebbb8da80ace7ca6adcdb0c66726f993659d
      a9a97386
    • M
      More Makefile Cleanup (#7097) · c7c7b07f
      mrambacher 提交于
      Summary:
      Cleans up some of the dependencies on test code in the Makefile while building tools:
      - Moves the test::RandomString, DBBaseTest::RandomString into Random
      - Moves the test::RandomHumanReadableString into Random
      - Moves the DestroyDir method into file_utils
      - Moves the SetupSyncPointsToMockDirectIO into sync_point.
      - Moves the FaultInjection Env and FS classes under env
      
      These changes allow all of the tools to build without dependencies on test_util, thereby simplifying the build dependencies.  By moving the FaultInjection code, the dependency in db_stress on different libraries for debug vs release was eliminated.
      
      Tested both release and debug builds via Make and CMake for both static and shared libraries.
      
      More work remains to clean up how the tools are built and remove some unnecessary dependencies.  There is also more work that should be done to get the Makefile and CMake to align in their builds -- what is in the libraries and the sizes of the executables are different.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7097
      
      Reviewed By: riversand963
      
      Differential Revision: D22463160
      
      Pulled By: pdillinger
      
      fbshipit-source-id: e19462b53324ab3f0b7c72459dbc73165cc382b2
      c7c7b07f
    • A
      save key comparisons in BlockIter::BinarySeek (#7068) · 82611ee2
      Andrew Kryczka 提交于
      Summary:
      This is a followup to https://github.com/facebook/rocksdb/issues/6646. In that PR, for simplicity I just appended a comparison against the 0th restart key in case `BinarySeek()`'s binary search landed at index 0. As a result there were `2/(N+1) + log_2(N)` key comparisons. This PR does it differently. Now we expand the binary search range by one so it also covers the case where target is at or before the restart key at index 0. As a result, it involves `log_2(N+1)` key comparisons.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7068
      
      Test Plan:
      ran readrandom with mostly default settings and counted key comparisons
      using `PerfContext`.
      
      before: `user_key_comparison_count = 28881965`
      after: `user_key_comparison_count = 27823245`
      
      setup command:
      
      ```
      $ TEST_TMPDIR=/dev/shm/dbbench ./db_bench -benchmarks=fillrandom,compact -write_buffer_size=1048576 -target_file_size_base=1048576 -max_bytes_for_level_base=4194304 -max_background_jobs=12 -level_compaction_dynamic_level_bytes=true -num=10000000
      ```
      
      benchmark command:
      
      ```
      $ TEST_TMPDIR=/dev/shm/dbbench/ ./db_bench -use_existing_db=true -benchmarks=readrandom -disable_auto_compactions=true -num=10000000 -compression_type=none -reads=1000000 -perf_level=3
      ```
      
      Reviewed By: anand1976
      
      Differential Revision: D22357032
      
      Pulled By: ajkr
      
      fbshipit-source-id: 8b01e9c1c2a4e9d02fc9dfe16c1cc0327f8bdf24
      82611ee2
    • Y
      Parameterize a few tests in DBWALTest (#7105) · f70ad031
      Yanqin Jin 提交于
      Summary:
      As title. The goal is to shorten the execution time of several tests
      when they are combined together in a single TEST_F.
      
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/7105
      
      Test Plan:
      make db_wal_test
      ./db_wal_test
      
      Reviewed By: ltamasi
      
      Differential Revision: D22442705
      
      Pulled By: riversand963
      
      fbshipit-source-id: 0ad49b8f21fa86dcd5a4d3c9a06af313735ac217
      f70ad031