- 16 3月, 2021 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: As title. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8054 Test Plan: make check Reviewed By: mrambacher Differential Revision: D27017955 Pulled By: riversand963 fbshipit-source-id: 829497d507bc89afbe982f8a8cf3555e52fd7098
-
- 10 3月, 2021 1 次提交
-
-
由 Ed rodriguez 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7994 Reviewed By: jay-zhuang Differential Revision: D26904603 Pulled By: ajkr fbshipit-source-id: 0af92a51de895b40c7faaa4f0870b3f63279fe21
-
- 26 2月, 2021 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: Allow applications to implement a custom compaction filter and pass it to BlobDB. The compaction filter's custom logic can operate on blobs. To do so, application needs to subclass `CompactionFilter` abstract class and implement `FilterV2()` method. Optionally, a method called `ShouldFilterBlobByKey()` can be implemented if application's custom logic rely solely on the key to make a decision without reading the blob, thus saving extra IO. Examples can be found in db/blob/db_blob_compaction_test.cc. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7974 Test Plan: make check Reviewed By: ltamasi Differential Revision: D26509280 Pulled By: riversand963 fbshipit-source-id: 59f9ae5614c4359de32f4f2b16684193cc537b39
-
- 23 2月, 2021 1 次提交
-
-
由 Akanksha Mahajan 提交于
Summary: Extend VerifyFileChecksums API to verify blob files in case of use_file_checksum. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7979 Test Plan: New unit test db_blob_corruption_test Reviewed By: ltamasi Differential Revision: D26534040 Pulled By: akankshamahajan15 fbshipit-source-id: 7dc5951a3df9d265ea1265e0122b43c966856ade
-
- 22 2月, 2021 1 次提交
-
-
由 mrambacher 提交于
Summary: I noticed tests frequently timing out on CircleCI when I submit a PR. I did some investigation and found the SeqAdvanceConcurrentTest suite (OneWriteQueue, TwoWriteQueues) tests were all taking a long time to complete (30 tests each taking at least 15K ms). This PR adds those test to the "slow reg" list in order to move them earlier in the execution sequence so that they are not the "long tail". For completeness, other tests that were also slow are: NumLevels/DBTestUniversalCompaction.UniversalCompactionTrivialMoveTest : 12 tests all taking 12K+ ms ReadSequentialFileTest with ReadaheadSize: 8 tests all 12K+ ms WriteUnpreparedTransactionTest.RecoveryTest : 2 tests at 22K+ ms DBBasicTest.EmptyFlush: 1 test at 35K+ ms RateLimiterTest.Rate: 1 test at 23K+ ms BackupableDBTest.ShareTableFilesWithChecksumsTransition: 1 test at 16K+ ms MulitThreadedDBTest.MultitThreaded: 78 tests at 10K+ ms TransactionStressTest.DeadlockStress: 7 tests at 11K+ ms DBBasicTestDeadline.IteratorDeadline: 3 tests at 10K+ ms No effort was made to determine why the tests were slow. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7973 Reviewed By: jay-zhuang Differential Revision: D26519130 Pulled By: mrambacher fbshipit-source-id: 11555c9115acc207e45e210a7fc7f879170a3853
-
- 11 2月, 2021 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Added support for detecting plugins linked in the "plugin/" directory and building them from our Makefile in a standardized way. See "plugin/README.md" for details. An example of a plugin that can be built in this way can be found in https://github.com/ajkr/dedupfs. There will be more to do in terms of making this process more convenient and adding support for CMake. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7918 Test Plan: my own plugin (https://github.com/ajkr/dedupfs) and also heard this patch worked with ZenFS. Reviewed By: pdillinger Differential Revision: D26189969 Pulled By: ajkr fbshipit-source-id: 6624d4357d0ffbaedb42f0d12a3fcb737c78f758
-
- 10 2月, 2021 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: Recent Github actions of format checking fail due to invalid location from where clang-format-diff.py is downloaded. Update the path to point to a stable, archived location. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7944 Test Plan: manually check the result of Github action. Reviewed By: ltamasi Differential Revision: D26345066 Pulled By: riversand963 fbshipit-source-id: 2b1a58c2e59c2f1eb11202d321d2ea002cb0917e
-
- 02 2月, 2021 1 次提交
-
-
由 mrambacher 提交于
Summary: (Fixes a regression introduced in the build_version generation PR https://github.com/facebook/rocksdb/issues/7866 ) In the Makefile case, needed to ignore stderr on the tag (everywhere else was fine). In the CMAKE case, no GIT implies "changes" so that we use the system date rather than the empty GIT date. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7916 Test Plan: Built in a tree that did not contain the ".git" directory. Validated that no errors appeared during the build process and that the build version date was not empty. Reviewed By: jay-zhuang Differential Revision: D26169203 Pulled By: mrambacher fbshipit-source-id: 3288a23b48d97efed5e5b38c9aefb3ef1153fa16
-
- 30 1月, 2021 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This PR adds the foundation classes for key-value integrity protection and the first use case: protecting live updates from the source buffers added to `WriteBatch` through the destination buffer in `MemTable`. The width of the protection info is not yet configurable -- only eight bytes per key is supported. This PR allows users to enable protection by constructing `WriteBatch` with `protection_bytes_per_key == 8`. It does not yet expose a way for users to get integrity protection via other write APIs (e.g., `Put()`, `Merge()`, `Delete()`, etc.). The foundation classes (`ProtectionInfo.*`) embed the coverage info in their type, and provide `Protect.*()` and `Strip.*()` functions to navigate between types with different coverage. For making bytes per key configurable (for powers of two up to eight) in the future, these classes are templated on the unsigned integer type used to store the protection info. That integer contains the XOR'd result of hashes with independent seeds for all covered fields. For integer fields, the hash is computed on the raw unadjusted bytes, so the result is endian-dependent. The most significant bytes are truncated when the hash value (8 bytes) is wider than the protection integer. When `WriteBatch` is constructed with `protection_bytes_per_key == 8`, we hold a `ProtectionInfoKVOTC` (i.e., one that covers key, value, optype aka `ValueType`, timestamp, and CF ID) for each entry added to the batch. The protection info is generated from the original buffers passed by the user, as well as the original metadata generated internally. When writing to memtable, each entry is transformed to a `ProtectionInfoKVOTS` (i.e., dropping coverage of CF ID and adding coverage of sequence number), since at that point we know the sequence number, and have already selected a memtable corresponding to a particular CF. This protection info is verified once the entry is encoded in the `MemTable` buffer. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7748 Test Plan: - an integration test to verify a wide variety of single-byte changes to the encoded `MemTable` buffer are caught - add to stress/crash test to verify it works in variety of configs/operations without intentional corruption - [deferred] unit tests for `ProtectionInfo.*` classes for edge cases like KV swap, `SliceParts` and `Slice` APIs are interchangeable, etc. Reviewed By: pdillinger Differential Revision: D25754492 Pulled By: ajkr fbshipit-source-id: e481bac6c03c2ab268be41359730f1ceb9964866
-
- 29 1月, 2021 1 次提交
-
-
由 mrambacher 提交于
Summary: Closes https://github.com/facebook/rocksdb/issues/7035 Changed how build_version.cc was generated: - Included the GIT tag/branch in the build_version file - Changed the "Build Date" to be: - If the GIT branch is "clean" (no changes), the date of the last git commit - If the branch is not clean, the current date - Added APIs to access the "build information", rather than accessing the strings directly. The build_version.cc file is now regenerated whenever the library objects are rebuilt. Verified that the built files remain the same size across builds on a "clean build" and the same information is reported by sst_dump --version Pull Request resolved: https://github.com/facebook/rocksdb/pull/7866 Reviewed By: pdillinger Differential Revision: D26086565 Pulled By: mrambacher fbshipit-source-id: 6fcbe47f6033989d5cf26a0ccb6dfdd9dd239d7f
-
- 14 1月, 2021 1 次提交
-
-
由 Adam Retter 提交于
Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
-
- 08 1月, 2021 1 次提交
-
-
由 mrambacher 提交于
Summary: Fixed the following to now pass ASC checks: * `ttl_test` * `blob_db_test` * `backupable_db_test`, * `delete_scheduler_test` Pull Request resolved: https://github.com/facebook/rocksdb/pull/7834 Reviewed By: jay-zhuang Differential Revision: D25795398 Pulled By: ajkr fbshipit-source-id: a10037817deda4fc7cbb353a2e00b62ed89b6476
-
- 07 1月, 2021 1 次提交
-
-
由 Adam Retter 提交于
Summary: Third batch of adding more tests to ASSERT_STATUS_CHECKED. * db_compaction_filter_test * db_compaction_test * db_dynamic_level_test * db_inplace_update_test * db_sst_test * db_tailing_iter_test * db_io_failure_test Also update GetApproximateSizes APIs to all return Status. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7715 Reviewed By: jay-zhuang Differential Revision: D25806896 Pulled By: pdillinger fbshipit-source-id: 6cb9d62ba5a756c645812754c596ad3995d7c262
-
- 05 1月, 2021 1 次提交
-
-
由 Adam Retter 提交于
Summary: Updates LZ4 and ZStd to the latest versions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7804 Reviewed By: ajkr Differential Revision: D25733770 Pulled By: pdillinger fbshipit-source-id: ea74ef9eecb57fc47934ef1d4ff950c99ddd5158
-
- 31 12月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: This fixes an issue introduced in https://github.com/facebook/rocksdb/pull/7769 that caused many errors about missing compression libraries to be displayed during compilation, although compilation actually succeeded. This PR fixes the compilation so the compression libraries are only introduced where strictly needed. It likely needs to be merged into the same branches as https://github.com/facebook/rocksdb/pull/7769 which I think are: 1. master 2. 6.15.fb 3. 6.16.fb Pull Request resolved: https://github.com/facebook/rocksdb/pull/7803 Reviewed By: ramvadiv Differential Revision: D25733743 Pulled By: pdillinger fbshipit-source-id: 6c04f6864b2ff4a345841d791a89b19e0e3f5bf7
-
- 24 12月, 2020 1 次提交
-
-
由 mrambacher 提交于
Summary: Added "no-elide-constructors to the ASSERT_STATUS_CHECK builds. This flag gives more errors/warnings for some of the Status checks where an inner class checks a Status and later returns it. In this case, without the elide check on, the returned status may not have been checked in the caller, thereby bypassing the checked code. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7798 Reviewed By: jay-zhuang Differential Revision: D25680451 Pulled By: pdillinger fbshipit-source-id: c3f14ed9e2a13f0a8c54d839d5fb4d1fc1e93917
-
- 23 12月, 2020 4 次提交
-
-
由 mrambacher 提交于
Summary: Added 10 more tests that pass the ASSERT_STATUS_CHECKED test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7524 Reviewed By: akankshamahajan15 Differential Revision: D24323093 Pulled By: ajkr fbshipit-source-id: 28d4106d0ca1740c3b896c755edf82d504b74801
-
由 Sergei Petrunia 提交于
Summary: Range Locking - an implementation based on the locktree library - Add a RangeTreeLockManager and RangeTreeLockTracker which implement range locking using the locktree library. - Point locks are handled as locks on single-point ranges. - Add a unit test: range_locking_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7506 Reviewed By: akankshamahajan15 Differential Revision: D25320703 Pulled By: cheng-chang fbshipit-source-id: f86347384b42ba2b0257d67eca0f45f806b69da7
-
由 Adam Retter 提交于
Summary: Fourth batch of adding more tests to ASSERT_STATUS_CHECKED. * db_range_del_test * db_write_test * random_access_file_reader_test * merge_test * external_sst_file_test * write_buffer_manager_test * stringappend_test * deletefile_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7718 Reviewed By: pdillinger Differential Revision: D25671608 fbshipit-source-id: 687a794e98a9e0cd5428ead9898ef05ced987c31
-
由 Akanksha Mahajan 提交于
Summary: add io_tracer_parser_test and prefetch_test under ASSERT_STATUS_CHECKED Pull Request resolved: https://github.com/facebook/rocksdb/pull/7793 Test Plan: ASSERT_STATUS_CHECKED=1 make check -j64 Reviewed By: jay-zhuang Differential Revision: D25673464 Pulled By: akankshamahajan15 fbshipit-source-id: 50e0b6f17160ddda206a521a7b47ee33e699a2d4
-
- 19 12月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: * Compatibility with older GCC. * Compatibility with older jemalloc libraries. * Remove Docker warning when building i686 binaries. * Fix case inconsistency in Java API naming (potential update to HISTORY.md deferred) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7776 Reviewed By: akankshamahajan15 Differential Revision: D25607235 Pulled By: pdillinger fbshipit-source-id: 7ab0fb7fa7a34e97ed0bec991f5081acb095777d
-
- 17 12月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
-
- 10 12月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: Second batch of adding more tests to ASSERT_STATUS_CHECKED. * external_sst_file_basic_test * checkpoint_test * db_wal_test * db_block_cache_test * db_logical_block_size_cache_test * db_blob_index_test * optimistic_transaction_test * transaction_test * point_lock_manager_test * write_prepared_transaction_test * write_unprepared_transaction_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7698 Reviewed By: cheng-chang Differential Revision: D25441664 Pulled By: pdillinger fbshipit-source-id: 9e78867f32321db5d4833e95eb96c5734526ef00
-
- 09 12月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: First batch of adding more tests to ASSERT_STATUS_CHECKED. * db_iterator_test * db_memtable_test * db_merge_operator_test * db_merge_operand_test * write_batch_test * write_batch_with_index_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7679 Reviewed By: ajkr Differential Revision: D25399270 Pulled By: pdillinger fbshipit-source-id: 3017d0a686aec5cd2d743fc2acbbf75df239f3ba
-
- 18 11月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: Expands on https://github.com/facebook/rocksdb/pull/7016 so that when `PORTABLE=1` is set the dependencies for RocksJava static target will also be built with backwards compatibility for MacOS as far back as 10.12 (i.e. 2016). Pull Request resolved: https://github.com/facebook/rocksdb/pull/7683 Reviewed By: ajkr Differential Revision: D25034164 Pulled By: pdillinger fbshipit-source-id: dc9e51828869ed9ec336a8a86683e4d0bfe04f27
-
- 12 11月, 2020 1 次提交
-
-
由 mrambacher 提交于
Summary: The Customizable class is an extension of the Configurable class and allows instances to be created by a name/ID. Classes that extend customizable can define their Type (e.g. "TableFactory", "Cache") and a method to instantiate them (TableFactory::CreateFromString). Customizable objects can be registered with the ObjectRegistry and created dynamically. Future PRs will make more types of objects extend Customizable. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6590 Reviewed By: cheng-chang Differential Revision: D24841553 Pulled By: zhichao-cao fbshipit-source-id: d0c2132bd932e971cbfe2c908ca2e5db30c5e155
-
- 11 11月, 2020 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: Since the hashes should not be persisted in output_validator nor mock_env. Also updated NPHash64 to use 64-bit seed, and comments. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7632 Test Plan: make check, and new build setting that enables modification to NPHash64, to check for behavior depending on specific values. Added that setting to one of the CircleCI configurations. Reviewed By: jay-zhuang Differential Revision: D24833780 Pulled By: pdillinger fbshipit-source-id: 02a57652ccf1ac105fbca79e77875bb7bf7c071f
-
- 26 10月, 2020 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: This is intended as the first commit toward a near-optimal alternative to static Bloom filters for SSTs. Stephan Walzer and I have agreed upon the name "Ribbon" for a PHSF based on his linear system construction in "Efficient Gauss Elimination for Near-Quadratic Matrices with One Short Random Block per Row, with Applications" ("SGauss") and my much faster "on the fly" algorithm for gaussian elimination (or for this linear system, "banding"), which can be faster than peeling while also more compact and flexible. See util/ribbon_alg.h for more detailed introduction and background. RIBBON = Rapid Incremental Boolean Banding ON-the-fly This commit just adds generic (templatized) core algorithms and a basic unit test showing some features, including the ability to construct structures within 2.5% space overhead vs. information theoretic lower bound. (Compare to cache-local Bloom filter's ~50% space overhead -> ~30% reduction anticipated.) This commit does not include the storage scheme necessary to make queries fast, especially for filter queries, nor fractional "result bits", but there is some description already and those implementations will come soon. Nor does this commit add FilterPolicy support, for use in SST files, but that will also come soon. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7491 Reviewed By: jay-zhuang Differential Revision: D24517954 Pulled By: pdillinger fbshipit-source-id: 0119ee597e250d7e0edd38ada2ba50d755606fa7
-
- 20 10月, 2020 1 次提交
-
-
由 Cheng Chang 提交于
Summary: In order to be able to introduce more locking protocols, we need to abstract out the locking subsystem in TransactionDB into a set of interfaces. PR https://github.com/facebook/rocksdb/pull/7013 introduces interface `LockTracker`. This PR is a follow up to take the first step to abstract out a `LockManager` interface. Further modifications to the interface may be needed when introducing the first implementation of range lock. But the idea here is to put the range lock implementation based on range tree under the `utilities/transactions/lock/range/range_tree`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7532 Test Plan: point_lock_manager_test Reviewed By: ajkr Differential Revision: D24238731 Pulled By: cheng-chang fbshipit-source-id: 2a9458cd8b3fb008d9529dbc4d3b28c24631f463
-
- 16 10月, 2020 1 次提交
-
-
由 Levi Tamasi 提交于
Summary: The patch adds blob file support to the `Get` API by extending `Version` so that whenever a blob reference is read from a file, the blob is retrieved from the corresponding blob file and passed back to the caller. (This is assuming the blob reference is valid and the blob file is actually part of the given `Version`.) It also introduces a cache of `BlobFileReader`s called `BlobFileCache` that enables sharing `BlobFileReader`s between callers. `BlobFileCache` uses the same backing cache as `TableCache`, so `max_open_files` (if specified) limits the total number of open (table + blob) files. TODO: proactively open/cache blob files and pin the cache handles of the readers in the metadata objects similarly to what `VersionBuilder::LoadTableHandlers` does for table files. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7540 Test Plan: `make check` Reviewed By: riversand963 Differential Revision: D24260219 Pulled By: ltamasi fbshipit-source-id: a8a2a4f11d3d04d6082201b52184bc4d7b0857ba
-
- 15 10月, 2020 1 次提交
-
-
由 Adam Retter 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7560 Reviewed By: akankshamahajan15 Differential Revision: D24323048 Pulled By: zhichao-cao fbshipit-source-id: 2094ef089c37f570d4ea30d28d2f46f9fa1ce0f1
-
- 14 10月, 2020 1 次提交
-
-
由 Zhichao Cao 提交于
Summary: Add plain_table_db_test to ASSERT_STATUS_CHECKED list Pull Request resolved: https://github.com/facebook/rocksdb/pull/7482 Test Plan: ASSERT_STATUS_CHECKED=1 make -j48 plain_table_db_test Reviewed By: riversand963 Differential Revision: D24034987 Pulled By: zhichao-cao fbshipit-source-id: e61c937d55ded0947cc8936937362dafed572a60
-
- 13 10月, 2020 1 次提交
-
-
由 Zhichao Cao 提交于
Summary: Added status check enforcement for db_flush_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7476 Test Plan: ASSERT_STATUS_CHECKED=1 make -j48 db_flush_test Reviewed By: akankshamahajan15 Differential Revision: D24033752 Pulled By: zhichao-cao fbshipit-source-id: d957934e1666d0043bebdd8a4149e94cdcbbb89b
-
- 09 10月, 2020 1 次提交
-
-
由 Akanksha Mahajan 提交于
Summary: Add status enforcement for following tests: 1. import_column_family_test 2. memory_test 3. table_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7500 Reviewed By: zhichao-cao Differential Revision: D24095887 Pulled By: akankshamahajan15 fbshipit-source-id: db8e1ec595852df143fad78a0c07bfdd27dc3c84
-
- 08 10月, 2020 2 次提交
-
-
由 Zhichao Cao 提交于
Summary: Add ldb_cmd_test to ASSERT_STATUS_CHECKED list Pull Request resolved: https://github.com/facebook/rocksdb/pull/7499 Test Plan: pass ASSERT_STATUS_CHECKED=1 make -j48 ldb_cmd_test Reviewed By: cheng-chang Differential Revision: D24086203 Pulled By: zhichao-cao fbshipit-source-id: 29592202b1d4335e566de15e7937269d98d57841
-
由 Levi Tamasi 提交于
Summary: The patch adds a class called `BlobFileReader` that can be used to retrieve blobs using the information available in blob references (e.g. blob file number, offset, and size). This will come in handy when implementing blob support for `Get`, `MultiGet`, and iterators, and also for compaction/garbage collection. When a `BlobFileReader` object is created (using the factory method `Create`), it first checks whether the specified file is potentially valid by comparing the file size against the combined size of the blob file header and footer (files smaller than the threshold are considered malformed). Then, it opens the file, and reads and verifies the header and footer. The verification involves magic number/CRC checks as well as checking for unexpected header/footer fields, e.g. incorrect column family ID or TTL blob files. Blobs can be retrieved using `GetBlob`. `GetBlob` validates the offset and compression type passed by the caller (because of the presence of the header and footer, the specified offset cannot be too close to the start/end of the file; also, the compression type has to match the one in the blob file header), and retrieves and potentially verifies and uncompresses the blob. In particular, when `ReadOptions::verify_checksums` is set, `BlobFileReader` reads the blob record header as well (as opposed to just the blob itself) and verifies the key/value size, the key itself, as well as the CRC of the blob record header and the key/value pair. In addition, the patch exposes the compression type from `BlobIndex` (both using an accessor and via `DebugString`), and adds a blob file read latency histogram to `InternalStats` that can be used with `BlobFileReader`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7461 Test Plan: `make check` Reviewed By: riversand963 Differential Revision: D23999219 Pulled By: ltamasi fbshipit-source-id: deb6b1160d251258b308d5156e2ec063c3e12e5e
-
- 07 10月, 2020 1 次提交
-
-
由 anand76 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/7460 Reviewed By: riversand963 Differential Revision: D24057636 Pulled By: anand1976 fbshipit-source-id: bfb13da6993a5e407be20073e4d6751dfb38e442
-
- 03 10月, 2020 3 次提交
-
-
由 Yanqin Jin 提交于
Summary: Enforce status check for corruption_test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7453 Test Plan: ``` ASSERT_STATUS_CHECKED=1 make corruption_test ./corruption_test ``` Reviewed By: jay-zhuang Differential Revision: D24006862 Pulled By: riversand963 fbshipit-source-id: 664677caf4c3007a25cf565cec3d677f2dcea130
-
由 sdong 提交于
Summary: Fix prefix_test so that it passes when ASSERT_STATUS_CHECKED=1 Pull Request resolved: https://github.com/facebook/rocksdb/pull/7495 Test Plan: Run the test with the option Reviewed By: anand1976 Differential Revision: D24069715 fbshipit-source-id: 54f74b58575a1b49dbdee9ea2d24751fa956b620
-
由 Zhichao Cao 提交于
Summary: Added status check enforcement for error_test_fs_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/7342 Test Plan: ASSERT_STATUS_CHECKED=1 make -j48 error_test_fs_test Reviewed By: akankshamahajan15 Differential Revision: D23972231 Pulled By: zhichao-cao fbshipit-source-id: fa41bfe440012e0c55f2c9507c1d0104e5e93f84
-