- 22 4月, 2021 1 次提交
-
-
由 Akanksha Mahajan 提交于
Summary: When WriteBufferManager is shared across DBs and column families to maintain memory usage under a limit, OOMs have been observed when flush cannot finish but writes continuously insert to memtables. In order to avoid OOMs, when memory usage goes beyond buffer_limit_ and DBs tries to write, this change will stall incoming writers until flush is completed and memory_usage drops. Design: Stall condition: When total memory usage exceeds WriteBufferManager::buffer_size_ (memory_usage() >= buffer_size_) WriterBufferManager::ShouldStall() returns true. DBImpl first block incoming/future writers by calling write_thread_.BeginWriteStall() (which adds dummy stall object to the writer's queue). Then DB is blocked on a state State::Blocked (current write doesn't go through). WBStallInterface object maintained by every DB instance is added to the queue of WriteBufferManager. If multiple DBs tries to write during this stall, they will also be blocked when check WriteBufferManager::ShouldStall() returns true. End Stall condition: When flush is finished and memory usage goes down, stall will end only if memory waiting to be flushed is less than buffer_size/2. This lower limit will give time for flush to complete and avoid continous stalling if memory usage remains close to buffer_size. WriterBufferManager::EndWriteStall() is called, which removes all instances from its queue and signal them to continue. Their state is changed to State::Running and they are unblocked. DBImpl then signal all incoming writers of that DB to continue by calling write_thread_.EndWriteStall() (which removes dummy stall object from the queue). DB instance creates WBMStallInterface which is an interface to block and signal DBs during stall. When DB needs to be blocked or signalled by WriteBufferManager, state_for_wbm_ state is changed accordingly (RUNNING or BLOCKED). Pull Request resolved: https://github.com/facebook/rocksdb/pull/7898 Test Plan: Added a new test db/db_write_buffer_manager_test.cc Reviewed By: anand1976 Differential Revision: D26093227 Pulled By: akankshamahajan15 fbshipit-source-id: 2bbd982a3fb7033f6de6153aa92a221249861aae
-
- 21 4月, 2021 4 次提交
-
-
由 Peter Dillinger 提交于
Summary: This partially reverts commit 10196d7e. The problem with this change is because of important filter use cases: FIFO compaction and SST writer. FIFO "compaction" always uses level 0 so would only use Ribbon filters if specifically including level 0 for the Ribbon filter policy. SST writer sets level_at_creation=-1 to indicate unknown level, and this would be treated the same as level 0 unless fixed. We are keeping the part about committing to permanent schema, which is only changes to API comments and HISTORY.md. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8212 Test Plan: CI Reviewed By: jay-zhuang Differential Revision: D27896468 Pulled By: pdillinger fbshipit-source-id: 50a775f7cba5d64fb729d9b982e355864020596e
-
由 Andrew Gallagher 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8208 Make include of "file_system.h" use the same include path as everywhere else. Reviewed By: riversand963, akankshamahajan15 Differential Revision: D27881606 fbshipit-source-id: fc1e076229fde21041a813c655ce017b5070c8b3
-
由 Andrew Kryczka 提交于
Summary: Fixes https://github.com/facebook/rocksdb/issues/6245. Adapted from https://github.com/facebook/rocksdb/issues/8201 and https://github.com/facebook/rocksdb/issues/8205. Previously we were writing the ingested file's smallest/largest internal keys with sequence number zero, or `kMaxSequenceNumber` in case of range tombstone. The former (sequence number zero) is incorrect and can lead to files being incorrectly ordered. The fix in this PR is to overwrite boundary keys that have sequence number zero with the ingested file's assigned sequence number. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8209 Test Plan: repro unit test Reviewed By: riversand963 Differential Revision: D27885678 Pulled By: ajkr fbshipit-source-id: 4a9f2c6efdfff81c3a9923e915ea88b250ee7b6a
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8210 Reviewed By: akankshamahajan15 Differential Revision: D27887612 Pulled By: ltamasi fbshipit-source-id: 0db8d0b6047334dc47fe30a98804449043454386
-
- 20 4月, 2021 5 次提交
-
-
由 Jay Zhuang 提交于
Summary: Unittest reports no space from time to time, which can be reproduced on a small memory machine with SHM. It's caused by large WAL files generated during the test, which is preallocated, but didn't truncate during close(). Adding the missing APIs to set preallocation. It added arm test as nightly build, as the test runs more than 1 hour. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8204 Test Plan: test on small memory arm machine Reviewed By: mrambacher Differential Revision: D27873145 Pulled By: jay-zhuang fbshipit-source-id: f797c429d6bc13cbcc673bc03fcc72adda55f506
-
由 Jay Zhuang 提交于
Summary: Moving ARM build from travis to CircleCI. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8203 Test Plan: CI Reviewed By: ajkr Differential Revision: D27861753 Pulled By: jay-zhuang fbshipit-source-id: 5e36a67f6fbb921c2ed80b284ba2de485411937b
-
由 Yanqin Jin 提交于
Summary: In a distributed environment, a file `rename()` operation can succeed on server (remote) side, but the client can somehow return non-ok status to RocksDB. Possible reasons include network partition, connection issue, etc. This happens in `rocksdb::SetCurrentFile()`, which can be called in `LogAndApply() -> ProcessManifestWrites()` if RocksDB tries to switch to a new MANIFEST. We currently always delete the new MANIFEST if an error occurs. This is problematic in distributed world. If the server-side successfully updates the CURRENT file via renaming, then a subsequent `DB::Open()` will try to look for the new MANIFEST and fail. As a fix, we can track the execution result of IO operations on the new MANIFEST. - If IO operations on the new MANIFEST fail, then we know the CURRENT must point to the original MANIFEST. Therefore, it is safe to remove the new MANIFEST. - If IO operations on the new MANIFEST all succeed, but somehow we end up in the clean up code block, then we do not know whether CURRENT points to the new or old MANIFEST. (For local POSIX-compliant FS, it should still point to old MANIFEST, but it does not matter if we keep the new MANIFEST.) Therefore, we keep the new MANIFEST. - Any future `LogAndApply()` will switch to a new MANIFEST and update CURRENT. - If process reopens the db immediately after the failure, then the CURRENT file can point to either the new MANIFEST or the old one, both of which exist. Therefore, recovery can succeed and ignore the other. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8192 Test Plan: make check Reviewed By: zhichao-cao Differential Revision: D27804648 Pulled By: riversand963 fbshipit-source-id: 9c16f2a5ce41bc6aadf085e48449b19ede8423e4
-
由 Levi Tamasi 提交于
Summary: Historically, the DB properties `rocksdb.cur-size-active-mem-table`, `rocksdb.cur-size-all-mem-tables`, and `rocksdb.size-all-mem-tables` called the method `MemTable::ApproximateMemoryUsage` for mutable memtables, which is not safe without synchronization. This resulted in data races with memtable inserts. The patch changes the code handling these properties to use `MemTable::ApproximateMemoryUsageFast` instead, which returns a cached value backed by an atomic variable. Two test cases had to be updated for this change. `MemoryTest.MemTableAndTableReadersTotal` was fixed by increasing the value size used so each value ends up in its own memtable, which was the original intention (note: the test has been broken in the sense that the test code didn't consider that memtable sizes below 64 KB get increased to 64 KB by `SanitizeOptions`, and has been passing only by accident). `DBTest.MemoryUsageWithMaxWriteBufferSizeToMaintain` relies on completely up-to-date values and thus was changed to use `ApproximateMemoryUsage` directly instead of going through the DB properties. Note: this should be safe in this case since there's only a single thread involved. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8206 Test Plan: `make check` Reviewed By: riversand963 Differential Revision: D27866811 Pulled By: ltamasi fbshipit-source-id: 7bd754d0565e0a65f1f7f0e78ffc093beef79394
-
由 Yanqin Jin 提交于
Summary: If `options.best_efforts_recovery == true`, RocksDB currently tolerates missing table files and recovers to the latest version without missing table files (not considering WAL). It is necessary to handle blob files as well to make the feature more complete. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8180 Test Plan: make check Reviewed By: ltamasi Differential Revision: D27840556 Pulled By: riversand963 fbshipit-source-id: 041685d0dc2e7779ac4f0374c07a8a327704aa5e
-
- 19 4月, 2021 1 次提交
-
-
由 Akanksha Mahajan 提交于
Summary: Test was flaky because for kUseDbSessionId naming, blob files use naming scheme kLegacyCrc32cAndFileSize. So expected number of files because of collision can vary. So disabling blobdb for this test case. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8197 Reviewed By: pdillinger Differential Revision: D27836997 Pulled By: akankshamahajan15 fbshipit-source-id: 5eb21a5f4acae3d6b730a9e1b207264fbc18cb80
-
- 17 4月, 2021 3 次提交
-
-
由 Akanksha Mahajan 提交于
Summary: Update release version to 6.20 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8199 Test Plan: No code change Reviewed By: ajkr Differential Revision: D27838750 Pulled By: akankshamahajan15 fbshipit-source-id: f02f722fc6bdd37d626d47a0e932bbecea3507a8
-
由 Peter Dillinger 提交于
Summary: Since the Ribbon filter schema seems good (compatible back to 6.15.0), this change commits to long term support of the SST schema, even though we expect the API for enabling Ribbon to change (still called NewExperimentalRibbonFilterPolicy). This also adds support for "hybrid" configuration in which some levels use Bloom (higher levels, lower numbered) for speed and the rest use Ribbon (lower levels, higher numbered) for memory space efficiency. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8198 Test Plan: unit test added, crash test support Reviewed By: jay-zhuang Differential Revision: D27831232 Pulled By: pdillinger fbshipit-source-id: 90e528677689474d293ed6710b42ba89fbd5b5ab
-
由 Adam Retter 提交于
Summary: The code for strcmp that was present does work when compiled for Windows unicode file paths. Needs backporting to: * 6.17.fb * 6.18.fb * 6.19.fb Pull Request resolved: https://github.com/facebook/rocksdb/pull/8190 Reviewed By: akankshamahajan15 Differential Revision: D27765588 Pulled By: jay-zhuang fbshipit-source-id: 89f8a5ac61fd7edc758340dfd335b0a5f96dae6e
-
- 16 4月, 2021 5 次提交
-
-
由 mrambacher 提交于
Summary: - Fixes the makefile to do the right thing when invoking multiple targets (e.g. make shared_lib install-shared). - Fixes the building of db_stress in shared lib mode. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8195 Reviewed By: pdillinger Differential Revision: D27803452 Pulled By: mrambacher fbshipit-source-id: 7c285d267770a359eb47f25855affdf58687e0e4
-
由 mrambacher 提交于
Summary: Added the Blob option settings from the AdvancedColmnFamilyOptions to the C API. There are no tests for getting/setting options in the C API currently, hence no specific test plans. Should there be a some? Pull Request resolved: https://github.com/facebook/rocksdb/pull/8148 Reviewed By: ltamasi Differential Revision: D27568495 Pulled By: mrambacher fbshipit-source-id: 3a52b784467ea2c4bc58be5f75c5d41f0a5c55d6
-
由 Akanksha Mahajan 提交于
Summary: Updated the test to wait until all trash files are deleted by SSTFileManager in the background. Since deletion runs in background so number of files deleted might not always be as expected. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8196 Reviewed By: jay-zhuang Differential Revision: D27812273 Pulled By: akankshamahajan15 fbshipit-source-id: d3ace1db34f91254b52fa455e09844d02801f58e
-
由 Akanksha Mahajan 提交于
Summary: Fix for failure to build in LITE mode on MacOs from BlobFileCompletionCallback unused private fields. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8189 Reviewed By: jay-zhuang Differential Revision: D27768341 Pulled By: akankshamahajan15 fbshipit-source-id: 14d31d7a9b52d308d9f9f27feff1977c5550622f
-
由 Akanksha Mahajan 提交于
Summary: Extend the DB::GetLiveFilesChecksumInfo API to blob files. This API is also used by the file_checksum_dump ldb command to dump checksum of SST files which now also dumps blob files checksum. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8179 Test Plan: Add new unit test Reviewed By: zhichao-cao Differential Revision: D27714965 Pulled By: akankshamahajan15 fbshipit-source-id: d8b7343ea845a64c83800336d88cced7152a8c92
-
- 15 4月, 2021 2 次提交
-
-
由 Yanqin Jin 提交于
Summary: As the name of `DBImpl::WriteLevel0TableForRecovery` suggests, the resulting table file should be placed on L0. However, the argument `level` passed to `BuildTable()` is -1. We need to correct this since the level information will be useful to determine file placement. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8187 Test Plan: make check Reviewed By: ltamasi Differential Revision: D27748570 Pulled By: riversand963 fbshipit-source-id: e1cd23128a8de31f14b1edc2ea92754c154e4f10
-
由 Justin Chapman 提交于
Summary: Resolves https://github.com/facebook/rocksdb/issues/8014 - Add an assertion on `DB::Open` to ensure `db_options.max_open_files` is unlimited if FIFO Compaction is being used. - This is to align with what the docs mention and to prevent premature data deletion. - Update tests to work with this assertion. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8172 Test Plan: ```bash $ make check -j$(nproc) Generated TARGETS Summary: - 6 libs - 0 binarys - 180 tests ``` Reviewed By: ajkr Differential Revision: D27768792 Pulled By: thejchap fbshipit-source-id: cf6350535e3a3577fec72bcba75b3c094dc7a6f3
-
- 14 4月, 2021 1 次提交
-
-
由 sdong 提交于
Summary: Add a blog post that calls for contribution in incremental compaction Pull Request resolved: https://github.com/facebook/rocksdb/pull/8182 Reviewed By: ajkr Differential Revision: D27724150 fbshipit-source-id: 42e474858b286a53e5aaa1c4e7242a8c745af651
-
- 13 4月, 2021 3 次提交
-
-
由 Yanqin Jin 提交于
Summary: Before this PR, `get_iostats_context()` will silently return a nullptr if no thread_local support is detected. This can be the result of build_detect_platform's failure to compile the simple code snippet on certain platforms, as reported in https://github.com/facebook/mysql-5.6/issues/904. To be safe, we should fail the compilation if user does not opt out IOStatsContext and ROCKSDB_SUPPORT_THREAD_LOCAL is not defined. If RocksDB relies on c++11, can we just always use thread_local? It turns out there might be performance concerns (https://github.com/facebook/rocksdb/issues/5774), which is beyond the scope of this PR. We can revisit this later. Here, we stick to the original impl. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8117 Reviewed By: ajkr Differential Revision: D27356847 Pulled By: riversand963 fbshipit-source-id: f7d5776842277598d8341b955febb601946801ae
-
由 Peter Dillinger 提交于
Summary: * CreateNewBackup(WithMetadata) returning the BackupID of new backup through optional new output param. This is especially useful with the new mutithreading support, so that you can transactionally determine the ID of a backup you create. * GetBackupInfo / GetLatestBackupInfo for individual backups, so that you don't have to comb through a vector of backups if you don't want to. Updated HISTORY.md (including re: BlobDB support as new feature) Pull Request resolved: https://github.com/facebook/rocksdb/pull/8170 Test Plan: Added test logic to existing tests, to minimize increase in cost of running tests Reviewed By: zhichao-cao Differential Revision: D27680410 Pulled By: pdillinger fbshipit-source-id: 1fc45b73d81aae293ccd4a43d9583d7fd915d3eb
-
由 Xavier Deguillard 提交于
Summary: When compiling RocksDB with Buck for ARM64, the linker complains about missing crc32 symbols that are defined in the crc32c_arm64.cc file. Since this file wasn't included in the build this is totally expected Pull Request resolved: https://github.com/facebook/rocksdb/pull/8168 Test Plan: The following no longer fails to link rocksdb: buck build mode/mac-xcode //eden/fs/service:edenfs#macosx-arm64 Reviewed By: zhichao-cao Differential Revision: D27664627 Pulled By: xavierd fbshipit-source-id: fb9d7a538599ee7a08882f87628731de6e641f8d
-
- 10 4月, 2021 2 次提交
-
-
由 Sahir Hoda 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8160 Reviewed By: riversand963 Differential Revision: D27672474 Pulled By: ajkr fbshipit-source-id: fdbbc3398f0b1d4cef6b68636e5caf369c34b3a7
-
由 David Carlier 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8169 Reviewed By: riversand963 Differential Revision: D27672457 Pulled By: ajkr fbshipit-source-id: b40a7ad5d09a754154f28c2574ef9f77c8a131bb
-
- 08 4月, 2021 4 次提交
-
-
由 Giuseppe Ottaviano 提交于
Summary: Current flush reason attribution is misleading or incorrect (depending on what the original intention was): - Flush due to WAL reaching its maximum size is attributed to `kWriteBufferManager` - Flushes due to full write buffer and write buffer manager are not distinguishable, both are attributed to `kWriteBufferFull` This changes the first to a new flush reason `kWALFull`, and splits the second between `kWriteBufferManager` and `kWriteBufferFull`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8150 Reviewed By: zhichao-cao Differential Revision: D27569645 Pulled By: ot fbshipit-source-id: 7e3c8ca186a6e71976e6b8e937297eebd4b769cc
-
由 Akanksha Mahajan 提交于
Summary: Enable backup/restore functionality with Integrated BlobDB in db_stress and crash test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8165 Test Plan: Ran python3 -u tools/db_crashtest.py --simple whitebox along with : 1. decreased "backup_in_one" value for backups to be more frequent and 2. manually changed code for "enable_blob_file" to be always true and apply blobdb params 100% for testing purpose. Reviewed By: ltamasi Differential Revision: D27636025 Pulled By: akankshamahajan15 fbshipit-source-id: 0d0e0d1479ced163f992872dc998e79c581bfc99
-
由 Akanksha Mahajan 提交于
Summary: Add support for blob files for backup/restore like table files. Since DB session ID is currently not supported for blob files (there is no place to store it in the header), so for blob files uses the kLegacyCrc32cAndFileSize naming scheme even if share_files_with_checksum_naming is set to kUseDbSessionId. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8129 Test Plan: Add new test units Reviewed By: ltamasi Differential Revision: D27408510 Pulled By: akankshamahajan15 fbshipit-source-id: b27434d189a639ef3e6ad165c61a143a2daaf06e
-
由 Peter Dillinger 提交于
Summary: Fixing another crash test failure in the case of write_dbid_to_manifest=true and reading a backup as read-only DB. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8164 Test Plan: enhanced unit test for backup as read-only DB, ran blackbox_crash_test more with elevated backup_one_in Reviewed By: zhichao-cao Differential Revision: D27622237 Pulled By: pdillinger fbshipit-source-id: 680d0f99ddb465a601737f2e3f2c80efd47384fb
-
- 07 4月, 2021 7 次提交
-
-
由 Peter Dillinger 提交于
Summary: Forgot to re-test crash test after adding read-only filesystem enforcement to https://github.com/facebook/rocksdb/issues/8142. The problem is ReadOnlyFileSystem would reject CreateDirIfMissing whenever DBOptions::create_if_missing=true. The fix that is better for users is to allow CreateDirIfMissing in ReadOnlyFileSystem if the directory exists, so that they don't cause a failure on using create_if_missing with opening backups as read-only DBs. Added this option test to the unit test (in addition to being in the crash test). Also fixed a couple of lints. And some better messaging from 'make format' so that when you run it with uncommitted changes, it's clear that it's only checking the uncommitted changes. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8161 Test Plan: local blackbox_crash_test with amplified backup_one_in Reviewed By: ajkr Differential Revision: D27614409 Pulled By: pdillinger fbshipit-source-id: 63ccb626c7e34c200d61c6bca2a8f60da9015179
-
由 Sahir Hoda 提交于
Summary: Updated instructions for installing zstd on CentOS and note about clang-format dependency Pull Request resolved: https://github.com/facebook/rocksdb/pull/8158 Reviewed By: riversand963 Differential Revision: D27598665 Pulled By: ajkr fbshipit-source-id: e349eeb91147f3163e170cc29c8460b06d739b5b
-
由 Peter Dillinger 提交于
Summary: A current limitation of backups is that you don't know the exact database state of when the backup was taken. With this new feature, you can at least inspect the backup's DB state without restoring it by opening it as a read-only DB. Rather than add something like OpenAsReadOnlyDB to the BackupEngine API, which would inhibit opening stackable DB implementations read-only (if/when their APIs support it), we instead provide a DB name and Env that can be used to open as a read-only DB. Possible follow-up work: * Add a version of GetBackupInfo for a single backup. * Let CreateNewBackup return the BackupID of the newly-created backup. Implementation details: Refactored ChrootFileSystem to split off new base class RemapFileSystem, which allows more general remapping of files. We use this base class to implement BackupEngineImpl::RemapSharedFileSystem. To minimize API impact, I decided to just add these fields `name_for_open` and `env_for_open` to those set by GetBackupInfo when include_file_details=true. Creating the RemapSharedFileSystem adds a bit to the memory consumption, perhaps unnecessarily in some cases, but this has been mitigated by (a) only initialize the RemapSharedFileSystem lazily when GetBackupInfo with include_file_details=true is called, and (b) using the existing `shared_ptr<FileInfo>` objects to hold most of the mapping data. To enhance API safety, RemapSharedFileSystem is wrapped by new ReadOnlyFileSystem which rejects any attempts to write. This uncovered a couple of places in which DB::OpenForReadOnly would write to the filesystem, so I fixed these. Added a release note because this affects logging. Additional minor refactoring in backupable_db.cc to support the new functionality. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8142 Test Plan: new test (run with ASAN and UBSAN), added to stress test and ran it for a while with amplified backup_one_in Reviewed By: ajkr Differential Revision: D27535408 Pulled By: pdillinger fbshipit-source-id: 04666d310aa0261ef6b2385c43ca793ce1dfd148
-
由 Yanqin Jin 提交于
Summary: According to https://github.com/facebook/rocksdb/issues/5907, each filter partition "should include the bloom of the prefix of the last key in the previous partition" so that SeekForPrev() in prefix mode can return correct result. The prefix of the last key in the previous partition does not necessarily have the same prefix as the first key in the current partition. Regardless of the first key in current partition, the prefix of the last key in the previous partition should be added. The existing code, however, does not follow this. Furthermore, there is another issue: when finishing current filter partition, `FullFilterBlockBuilder::AddPrefix()` is called for the first key in next filter partition, which effectively overwrites `last_prefix_str_` prematurely. Consequently, when the filter block builder proceeds to the next partition, `last_prefix_str_` will be the prefix of its first key, leaving no way of adding the bloom of the prefix of the last key of the previous partition. Prefix extractor is FixedLength.2. ``` [ filter part 1 ] [ filter part 2 ] abc d ``` When SeekForPrev("abcd"), checking the filter partition will land on filter part 2 because "abcd" > "abc" but smaller than "d". If the filter in filter part 2 happens to return false for the test for "ab", then SeekForPrev("abcd") will build incorrect iterator tree in non-total-order mode. Also fix a unit test which starts to fail following this PR. `InDomain` should not fail due to assertion error when checking on an arbitrary key. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8137 Test Plan: ``` make check ``` Without this fix, the following command will fail pretty soon. ``` ./db_stress --acquire_snapshot_one_in=10000 --avoid_flush_during_recovery=0 \ --avoid_unnecessary_blocking_io=0 --backup_max_size=104857600 --backup_one_in=0 \ --batch_protection_bytes_per_key=0 --block_size=16384 --bloom_bits=17 \ --bottommost_compression_type=disable --cache_index_and_filter_blocks=1 --cache_size=1048576 \ --checkpoint_one_in=0 --checksum_type=kxxHash64 --clear_column_family_one_in=0 \ --compact_files_one_in=1000000 --compact_range_one_in=1000000 --compaction_ttl=0 \ --compression_max_dict_buffer_bytes=0 --compression_max_dict_bytes=0 \ --compression_parallel_threads=1 --compression_type=zstd --compression_zstd_max_train_bytes=0 \ --continuous_verification_interval=0 --db=/dev/shm/rocksdb/rocksdb_crashtest_whitebox \ --db_write_buffer_size=8388608 --delpercent=5 --delrangepercent=0 --destroy_db_initially=0 --enable_blob_files=0 \ --enable_compaction_filter=0 --enable_pipelined_write=1 --file_checksum_impl=big --flush_one_in=1000000 \ --format_version=5 --get_current_wal_file_one_in=0 --get_live_files_one_in=1000000 --get_property_one_in=1000000 \ --get_sorted_wal_files_one_in=0 --index_block_restart_interval=4 --index_type=2 --ingest_external_file_one_in=0 \ --iterpercent=10 --key_len_percent_dist=1,30,69 --level_compaction_dynamic_level_bytes=True \ --log2_keys_per_lock=10 --long_running_snapshots=1 --mark_for_compaction_one_file_in=0 \ --max_background_compactions=20 --max_bytes_for_level_base=10485760 --max_key=100000000 --max_key_len=3 \ --max_manifest_file_size=1073741824 --max_write_batch_group_size_bytes=16777216 --max_write_buffer_number=3 \ --max_write_buffer_size_to_maintain=8388608 --memtablerep=skip_list --mmap_read=1 --mock_direct_io=False \ --nooverwritepercent=0 --open_files=500000 --ops_per_thread=20000000 --optimize_filters_for_memory=0 --paranoid_file_checks=1 --partition_filters=1 --partition_pinning=0 --pause_background_one_in=1000000 \ --periodic_compaction_seconds=0 --prefixpercent=5 --progress_reports=0 --read_fault_one_in=0 --read_only=0 \ --readpercent=45 --recycle_log_file_num=0 --reopen=20 --secondary_catch_up_one_in=0 \ --snapshot_hold_ops=100000 --sst_file_manager_bytes_per_sec=104857600 \ --sst_file_manager_bytes_per_truncate=0 --subcompactions=2 --sync=0 --sync_fault_injection=False \ --target_file_size_base=2097152 --target_file_size_multiplier=2 --test_batches_snapshots=0 --test_cf_consistency=0 \ --top_level_index_pinning=0 --unpartitioned_pinning=1 --use_blob_db=0 --use_block_based_filter=0 \ --use_direct_io_for_flush_and_compaction=0 --use_direct_reads=0 --use_full_merge_v1=0 --use_merge=0 \ --use_multiget=0 --use_ribbon_filter=0 --use_txn=0 --user_timestamp_size=8 --verify_checksum=1 \ --verify_checksum_one_in=1000000 --verify_db_one_in=100000 --write_buffer_size=4194304 \ --write_dbid_to_manifest=1 --writepercent=35 ``` Reviewed By: pdillinger Differential Revision: D27553054 Pulled By: riversand963 fbshipit-source-id: 60e391e4a2d8d98a9a3172ec5d6176b90ec3de98
-
由 sunby 提交于
Summary: Failures in `InvalidatePageCache` will change the API contract. So we remove the status check for `InvalidatePageCache` in `SstFileWriter::Add()`, `SstFileWriter::Finish` and `Rep::DeleteRange` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8156 Reviewed By: riversand963 Differential Revision: D27597012 Pulled By: ajkr fbshipit-source-id: 2872051695d50cc47ed0f2848dc582464c00076f
-
由 Yanqin Jin 提交于
Summary: Fixes https://github.com/facebook/rocksdb/issues/6548. If we do not reset the pinnable slice before calling get, we will see the following assertion failure while running the test with multiple column families. ``` db_bench: ./include/rocksdb/slice.h:168: void rocksdb::PinnableSlice::PinSlice(const rocksdb::Slice&, rocksdb::Cleanable*): Assertion `!pinned_' failed. ``` This happens in `BlockBasedTable::Get()`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8154 Test Plan: ./db_bench --benchmarks=fillseq -num_column_families=3 ./db_bench --benchmarks=readrandom -use_existing_db=1 -num_column_families=3 Reviewed By: ajkr Differential Revision: D27587589 Pulled By: riversand963 fbshipit-source-id: 7379e7649ba40f046d6a4014c9ad629cb3f9a786
-
由 Adam Retter 提交于
Summary: The previous version of ZStd doesn't build correctly with Make 3.82. Updating it resolves the issue. jay-zhuang This also needs to be cherry-picked to: 1. 6.17.fb 2. 6.18.fb 3. 6.19.fb Pull Request resolved: https://github.com/facebook/rocksdb/pull/8155 Reviewed By: riversand963 Differential Revision: D27596460 Pulled By: ajkr fbshipit-source-id: ac8492245e6273f54efcc1587346a797a91c9441
-
- 06 4月, 2021 2 次提交
-
-
由 shadowlux 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8149 Reviewed By: ajkr Differential Revision: D27571017 Pulled By: riversand963 fbshipit-source-id: a73427e907a7cb899debf55d60a2ede726695277
-
由 David Carlier 提交于
Summary: full name. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8144 Reviewed By: ajkr Differential Revision: D27581146 Pulled By: riversand963 fbshipit-source-id: 7d4cbde02a07aa4676e35aeb60c3d6f1f492a3cd
-