- 30 3月, 2021 3 次提交
-
-
由 kshair 提交于
Summary: terated -> treated Pull Request resolved: https://github.com/facebook/rocksdb/pull/7960 Reviewed By: ajkr Differential Revision: D26677005 Pulled By: zhichao-cao fbshipit-source-id: 6221305afb263aa60f674a4113aa30cb8f3914e6
-
由 mrambacher 提交于
Summary: The check in db_bench for table_cache_numshardbits was 0 < bits <= 20, whereas the check in LRUCache was 0 < bits < 20. Changed the two values to match to avoid a crash in db_bench on a null cache. Fixes https://github.com/facebook/rocksdb/issues/7393 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8110 Reviewed By: zhichao-cao Differential Revision: D27353522 Pulled By: mrambacher fbshipit-source-id: a414bd23b5bde1f071146b34cfca5e35c02de869
-
由 yaphet 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8118 Reviewed By: ajkr Differential Revision: D27367488 Pulled By: zhichao-cao fbshipit-source-id: 6ed598c74ab9232f2e56326b3a30476d473699d7
-
- 29 3月, 2021 4 次提交
-
-
由 mrambacher 提交于
Summary: Ran a spell check over the comments in the include/rocksdb directory and fixed any mis-spellings. There are still some variable names that are spelled incorrectly (like SizeApproximationOptions::include_memtabtles, SstFileMetaData::oldest_ancester_time) that were not fixed, as those would break compilation. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8120 Reviewed By: zhichao-cao Differential Revision: D27366034 Pulled By: mrambacher fbshipit-source-id: 6a3f3674890bb6acc751e9c5887a8fbb6adca5df
-
由 Yanqin Jin 提交于
Summary: Currently, partitioned filter does not support user-defined timestamp. Disable it for now in ts stress test so that the contrun jobs can proceed. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8127 Test Plan: make crash_test_with_ts Reviewed By: ajkr Differential Revision: D27388488 Pulled By: riversand963 fbshipit-source-id: 5ccff18121cb537bd82f2ac072cd25efb625c666
-
由 mrambacher 提交于
Summary: Because build_version.cc is dependent on the library objects (to force a re-generation of it), the library objects would be built in order to satisfy this rule. Because there is a build_version.d file, it would need generated and included. Change the ALL_DEPS/FILES to not include build_version.cc (meaning no .d file for it, which is okay since it is generated). Also changed the rule on whether or not to generate DEP files to skip tags. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8097 Reviewed By: ajkr Differential Revision: D27299815 Pulled By: mrambacher fbshipit-source-id: 1efbe8a56d062f57ae13b6c2944ad3faf775087e
-
由 anand76 提交于
Summary: Currently, we only truncate the latest alive WAL files when the DB is opened. If the latest WAL file is empty or was flushed during Open, its not truncated since the file will be deleted later on in the Open path. However, before deletion, a new WAL file is created, and if the process crash loops between the new WAL file creation and deletion of the old WAL file, the preallocated space will keep accumulating and eventually use up all disk space. To prevent this, always truncate the latest WAL file, even if its empty or the data was flushed. Tests: Add unit tests to db_wal_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/8122 Reviewed By: riversand963 Differential Revision: D27366132 Pulled By: anand1976 fbshipit-source-id: f923cc03ef033ccb32b140d36c6a63a8152f0e8e
-
- 27 3月, 2021 7 次提交
-
-
由 shadowlux 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8095 Reviewed By: riversand963 Differential Revision: D27318295 Pulled By: jay-zhuang fbshipit-source-id: a014fbd28fdd7a26648da19a766dc00d2de9fdc8
-
由 Jay Zhuang 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8119 Reviewed By: sushilpa Differential Revision: D27365407 Pulled By: jay-zhuang fbshipit-source-id: 327c09bf76834ce0be4287680640adc8b88bcec2
-
由 wolfkdy 提交于
Summary: see https://github.com/facebook/rocksdb/issues/7376. The `wfe` op on ARM platform is not suitable to relax CPU. Use `yield` op. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7438 Reviewed By: riversand963 Differential Revision: D24063427 Pulled By: jay-zhuang fbshipit-source-id: b0ebc5590d7555bd21b30f15cd59f84dc006367a
-
由 mrambacher 提交于
Summary: For some branches, I see an error during analyze on this code. I do not know why it is not persistent, but this should address the error: Logic error | Result of operation is garbage or undefined | trace_replay.cc | Replay | 436 | 30 | View Report DecodeCFAndKey(trace.payload, &get_payload.cf_id, &get_payload.get_key); -- 433 | } else { 434 | TracerHelper::DecodeGetPayload(&trace, &get_payload); | 25←Calling 'TracerHelper::DecodeGetPayload'→ | 25 | ← | Calling 'TracerHelper::DecodeGetPayload' | → 25 | ← | Calling 'TracerHelper::DecodeGetPayload' | → | 29←Returning from 'TracerHelper::DecodeGetPayload'→ | 29 | ← | Returning from 'TracerHelper::DecodeGetPayload' | → 29 | ← | Returning from 'TracerHelper::DecodeGetPayload' | → 435 | } 436 | if (get_payload.cf_id > 0 && | 30←The left operand of '>' is a garbage value | 30 | ← | The left operand of '>' is a garbage value 30 | ← | The left operand of '>' is a garbage value 437 | cf_map_.find(get_payload.cf_id) == cf_map_.end()) { 438 | return Status::Corruption("Invalid Column Family ID."); 439 | } Pull Request resolved: https://github.com/facebook/rocksdb/pull/8121 Reviewed By: zhichao-cao Differential Revision: D27366022 Pulled By: mrambacher fbshipit-source-id: 309c05dbab08cd7ab7f15389e8456f09196f37f6
-
由 anand76 提交于
Summary: The snapshot structure returned by rocksdb_transaction_get_snapshot is supposed to be freed by calling rocksdb_free(), so allocate using malloc rather than new. Fixes https://github.com/facebook/rocksdb/issues/6112 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8114 Reviewed By: akankshamahajan15 Differential Revision: D27362923 Pulled By: anand1976 fbshipit-source-id: e93a8b1ffe26dafbe22529907f72b796ae971214
-
由 Zhichao Cao 提交于
Summary: Remove disabled tests Pull Request resolved: https://github.com/facebook/rocksdb/pull/8123 Test Plan: make check Reviewed By: ltamasi Differential Revision: D27367066 Pulled By: zhichao-cao fbshipit-source-id: 71fa1d492d9b0144decff0a1d0e0ef25c0ecc4ba
-
由 junhan lee 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8088 Reviewed By: ajkr Differential Revision: D27270378 Pulled By: zhichao-cao fbshipit-source-id: 05af12c63855d00cc57bab9866fc8193c03a404e
-
- 26 3月, 2021 4 次提交
-
-
由 Levi Tamasi 提交于
Summary: The patch adds a resource management/RAII class called `ThreadGuard`, which can be used to ensure that the managed thread is joined when the `ThreadGuard` is destroyed, regardless of whether it is due to the object going out of scope, an early return, an exception etc. This is important because if an `std::thread` object is destroyed without having been joined (or detached) first, the process is aborted (via `std::terminate`). For now, `ThreadGuard` is only used in the test case `ExternalSSTFileTest.PickedLevelBug`; however, it could come in handy elsewhere in the codebase as well (both in test code and "real" code). Case in point: in the `PickedLevelBug` test case, with the earlier code we could end up in the above situation when the following assertion (which is before the threads are joined) is triggered: ``` ASSERT_FALSE(bg_compact_started.load()); ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8112 Test Plan: ``` make check gtest-parallel --repeat=10000 ./external_sst_file_test --gtest_filter="*PickedLevelBug" ``` Reviewed By: riversand963 Differential Revision: D27343185 Pulled By: ltamasi fbshipit-source-id: 2a8c3aa68bc78cc03ec0dbae909fb25c2cd15c69
-
由 Zhichao Cao 提交于
Summary: There is bug in the current code base introduced in https://github.com/facebook/rocksdb/issues/8049 , we still set the SST file write IO Error only case as hard error. Fix it by removing the logic. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8107 Test Plan: make check, error_handler_fs_test Reviewed By: anand1976 Differential Revision: D27321422 Pulled By: zhichao-cao fbshipit-source-id: c014afc1553ca66b655e3bbf9d0bf6eb417ccf94
-
由 storagezhang 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8066 Reviewed By: jay-zhuang Differential Revision: D27280799 Pulled By: mrambacher fbshipit-source-id: 68f91f5af4ffe0a84be581961bf9366887f47702
-
由 Andrew Kryczka 提交于
Summary: Previously it only applied to block-based tables generated by flush. This restriction was undocumented and blocked a new use case. Now compression sampling applies to all block-based tables we generate when it is enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8105 Test Plan: new unit test Reviewed By: riversand963 Differential Revision: D27317275 Pulled By: ajkr fbshipit-source-id: cd9fcc5178d6515e8cb59c6facb5ac01893cb5b0
-
- 25 3月, 2021 3 次提交
-
-
由 Jay Zhuang 提交于
Summary: `strerror()` is not thread-safe, using `strerror_r()` instead. The API could be different on the different platforms, used the code from https://github.com/facebook/folly/blob/0deef031cb8aab76dc7e736f8b7c22d701d5f36b/folly/String.cpp#L457 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8087 Reviewed By: mrambacher Differential Revision: D27267151 Pulled By: jay-zhuang fbshipit-source-id: 4b8856d1ec069d5f239b764750682c56e5be9ddb
-
由 Connor 提交于
Summary: **Summary:** When doing CompactFiles on the files of multiple levels(num_level > 2) with L0 is included, the compaction would fail like this. ![image](https://user-images.githubusercontent.com/13497871/109975371-8b601280-7d35-11eb-830f-f732dc1f9246.png) The reason is that in `VerifyCompactionFileConsistency` it checks the levels between the L0 and base level should be empty, but it regards the compaction triggered by `CompactFiles` as an L0 -> base level compaction wrongly. The condition is committed several years ago, whereas it isn't correct anymore. ```c++ if (vstorage->compaction_style_ == kCompactionStyleLevel && c->start_level() == 0 && c->num_input_levels() > 2U) ``` So this PR just deletes the incorrect check. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8024 Test Plan: make check Reviewed By: jay-zhuang Differential Revision: D26907060 Pulled By: ajkr fbshipit-source-id: 538cef32faf464cd422e3f8de236ea3e58880c2b
-
由 Yanqin Jin 提交于
Summary: As title. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8104 Test Plan: build_tools/rocksdb-lego-determinator stress_crash_with_ts Reviewed By: ltamasi Differential Revision: D27312265 Pulled By: riversand963 fbshipit-source-id: 3175a9d9074bdb282137c6518402d622436931d6
-
- 24 3月, 2021 7 次提交
-
-
由 Peter Dillinger 提交于
Summary: Improved handling of -bits_per_key other than 10, but at least the OptimizeForMemory test is simply not designed for generally handling other settings. (ribbon_test does have a statistical framework for this kind of testing, but it's not important to do that same for Bloom right now.) Closes https://github.com/facebook/rocksdb/issues/7019 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8093 Test Plan: for I in `seq 1 20`; do ./bloom_test --gtest_filter=-*OptimizeForMemory* --bits_per_key=$I &> /dev/null || echo FAILED; done Reviewed By: mrambacher Differential Revision: D27275875 Pulled By: pdillinger fbshipit-source-id: 7362e8ac2c41ea11f639412e4f30c8b375f04388
-
由 Akanksha Mahajan 提交于
Summary: Fix race condition in DBSSTTest.DBWithMaxSpaceAllowedWithBlobFiles where background flush thread updates delete_blob_file but in test thread Flush() already completes after getting bg_error and delete_blob_file remains false. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8092 Test Plan: Ran ASAN job few times on CircleCI Reviewed By: riversand963 Differential Revision: D27275815 Pulled By: akankshamahajan15 fbshipit-source-id: 2939ad1671403881573bbe07c71aa474c5019130
-
由 Zhichao Cao 提交于
Summary: Added append with checksum handoff API to hdfs Pull Request resolved: https://github.com/facebook/rocksdb/pull/8084 Test Plan: make check Reviewed By: ajkr Differential Revision: D27237823 Pulled By: zhichao-cao fbshipit-source-id: 93b38db23b1811a6daa049afb89240089ec6f67c
-
由 Yanqin Jin 提交于
Summary: As title. All core db implementations should stay in db_impl. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8082 Test Plan: make check Reviewed By: ajkr Differential Revision: D27211442 Pulled By: riversand963 fbshipit-source-id: e0953fde75064740e899aaff7989ff033b7f5232
-
由 Yanqin Jin 提交于
Summary: Fix the following error while running `make crash_test` ``` Traceback (most recent call last): File "tools/db_crashtest.py", line 705, in <module> main() File "tools/db_crashtest.py", line 696, in main blackbox_crash_main(args, unknown_args) File "tools/db_crashtest.py", line 479, in blackbox_crash_main + list({'db': dbname}.items())), unknown_args) File "tools/db_crashtest.py", line 414, in gen_cmd finalzied_params = finalize_and_sanitize(params) File "tools/db_crashtest.py", line 331, in finalize_and_sanitize dest_params.get("user_timestamp_size") > 0): TypeError: '>' not supported between instances of 'NoneType' and 'int' ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8091 Test Plan: make crash_test Reviewed By: ltamasi Differential Revision: D27268276 Pulled By: riversand963 fbshipit-source-id: ed2873b9587ecc51e24abc35ef2bd3d91fb1ed1b
-
由 Vlad Artamonov 提交于
Summary: This is a small fix to what I think is a mistype in two comments in `DBOptionsInterface.java`. If it was not an error, feel free to close. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8086 Reviewed By: ajkr Differential Revision: D27260488 Pulled By: mrambacher fbshipit-source-id: 469daadaf6039d5b5187132b8e0c7c3672842f21
-
由 Yanqin Jin 提交于
Summary: As title. Always specify namespace::symbol_name... Test plan CircleCI and other CI results Pull Request resolved: https://github.com/facebook/rocksdb/pull/8090 Reviewed By: ltamasi Differential Revision: D27256130 Pulled By: riversand963 fbshipit-source-id: b9b9ae2b3a8b4a16f0384292e71c6aecca93c570
-
- 23 3月, 2021 3 次提交
-
-
由 Yanqin Jin 提交于
Summary: Add some basic test for user-defined timestamp to db_stress. Currently, read with timestamp always tries to read using the current timestamp. Due to the per-key timestamp-sequence ordering constraint, we only add timestamp- related tests to the `NonBatchedOpsStressTest` since this test serializes accesses to the same key and uses a file to cross-check data correctness. The timestamp feature is not supported in a number of components, e.g. Merge, SingleDelete, DeleteRange, CompactionFilter, Readonly instance, secondary instance, SST file ingestion, transaction, etc. Therefore, db_stress should exit if user enables both timestamp and these features at the same time. The (currently) incompatible features can be found in `CheckAndSetOptionsForUserTimestamp`. This PR also fixes a bug triggered when timestamp is enabled together with `index_type=kBinarySearchWithFirstKey`. This bug fix will also be in another separate PR with more unit tests coverage. Fixing it here because I do not want to exclude the index type from crash test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8061 Test Plan: make crash_test_with_ts Reviewed By: jay-zhuang Differential Revision: D27056282 Pulled By: riversand963 fbshipit-source-id: c3e00ad1023fdb9ebbdf9601ec18270c5e2925a9
-
由 Levi Tamasi 提交于
Summary: Since our stress/crash tests by default generate values of size 8, 16, or 24, it does not make much sense to set `min_blob_size` to 256. The patch updates the set of potential `min_blob_size` values in the crash test script and in `db_stress` where it might be set dynamically using `SetOptions`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8085 Test Plan: Ran `make check` and tried the crash test script. Reviewed By: riversand963 Differential Revision: D27238620 Pulled By: ltamasi fbshipit-source-id: 4a96f9944b1ed9220d3045c5ab0b34c49009aeee
-
由 Yanqin Jin 提交于
Summary: The implementation of TransactionDB::WrapDB() and TransactionDB::WrapStackableDB() are almost identical, except for the type of the first argument `db`. This PR adds a new template function in anonymous namespace, and calls it in the above two functions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8079 Test Plan: make check Reviewed By: lth Differential Revision: D27184575 Pulled By: riversand963 fbshipit-source-id: f2855a6db3a7e897d0d611f7050ca4b696c56a7a
-
- 22 3月, 2021 1 次提交
-
-
由 Zhichao Cao 提交于
Summary: Update release version to 6.19 Pull Request resolved: https://github.com/facebook/rocksdb/pull/8083 Test Plan: no code change Reviewed By: riversand963 Differential Revision: D27222083 Pulled By: zhichao-cao fbshipit-source-id: 94b49997019347e6e6a9e341837f4f9d3149428c
-
- 20 3月, 2021 5 次提交
-
-
由 Peter Dillinger 提交于
Summary: This does not add any new public APIs or published functionality, but adds the ability to read and use (and in tests, write) backups with a new meta file schema, based on the old schema but not forward-compatible (before this change). The new schema enables some capabilities not in the old: * Explicit versioning, so that users get clean error messages the next time we want to break forward compatibility. * Ignoring unrecognized fields (with warning), so that new non-critical features can be added without breaking forward compatibility. * Rejecting future "non-ignorable" fields, so that new features critical to some use-cases could potentially be added outside of linear schema versions, with broken forward compatibility. * Fields at the end of the meta file, such as for checksum of the meta file's contents (up to that point) * New optional 'size' field for each file, which is checked when present * Optionally omitting 'crc32' field, so that we aren't required to have a crc32c checksum for files to take a backup. (E.g. to support backup via hard links and to better support file custom checksums.) Because we do not have a JSON parser and to share code, the new schema is simply derived from the old schema. BackupEngine code is updated to allow missing checksums in some places, and to make that easier, `has_checksum` and `verify_checksum_after_work` are eliminated. Empty `checksum_hex` indicates checksum is unknown. I'm not too afraid of regressing on data integrity, because (a) we have pretty good test coverage of corruption detection in backups, and (b) we are increasingly relying on the DB itself for data integrity rather than it being an exclusive feature of backups. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8069 Test Plan: new unit tests, added to crash test (some local run with boosted backup probability) Reviewed By: ajkr Differential Revision: D27139824 Pulled By: pdillinger fbshipit-source-id: 9e0e4decfb42bb84783d64d2d246456d97e8e8c5
-
由 storagezhang 提交于
Summary: Remove unused variable `Slice blob_to_write` in `db/blob/blob_file_cache_test.cc`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8067 Reviewed By: zhichao-cao Differential Revision: D27107693 Pulled By: riversand963 fbshipit-source-id: 9bfd4d296a6a1714ad5c1fa5bb231a0c52dbd56d
-
由 storagezhang 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8068 Reviewed By: zhichao-cao Differential Revision: D27147685 Pulled By: riversand963 fbshipit-source-id: 5428b1c0142ecae17c977fba31a6d49b52983d1c
-
由 storagezhang 提交于
Summary: switch may not cover all branch in `db/c.cc`: ```c++ void rocksdb_options_set_access_hint_on_compaction_start( rocksdb_options_t* opt, int v) { switch(v) { case 0: opt->rep.access_hint_on_compaction_start = ROCKSDB_NAMESPACE::Options::NONE; break; case 1: opt->rep.access_hint_on_compaction_start = ROCKSDB_NAMESPACE::Options::NORMAL; break; case 2: opt->rep.access_hint_on_compaction_start = ROCKSDB_NAMESPACE::Options::SEQUENTIAL; break; case 3: opt->rep.access_hint_on_compaction_start = ROCKSDB_NAMESPACE::Options::WILLNEED; break; } } ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8065 Reviewed By: riversand963 Differential Revision: D27102892 Pulled By: zhichao-cao fbshipit-source-id: ad1d20d192712878e61597311ba75b55df0066d7
-
由 Zhichao Cao 提交于
Summary: Add the new Append and PositionedAppend API to env WritableFile. User is able to benefit from the write checksum handoff API when using the legacy Env classes. FileSystem already implemented the checksum handoff API. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8071 Test Plan: make check, added new unit test. Reviewed By: anand1976 Differential Revision: D27177043 Pulled By: zhichao-cao fbshipit-source-id: 430c8331fc81099fa6d00f4fff703b68b9e8080e
-
- 19 3月, 2021 3 次提交
-
-
由 Yanqin Jin 提交于
Summary: As title. Test plan ASSERT_STATUS_CHECKED=1 make -j20 backupable_db_test error_handler_fs_test ./backupable_db_test ./error_handler_fs_test Pull Request resolved: https://github.com/facebook/rocksdb/pull/8075 Reviewed By: zhichao-cao Differential Revision: D27173832 Pulled By: riversand963 fbshipit-source-id: 37dac50f7c89127804ff2572abddd4174642de30
-
由 Yanqin Jin 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8077 Test Plan: Travis CI Reviewed By: zhichao-cao Differential Revision: D27178276 Pulled By: riversand963 fbshipit-source-id: 17911dcc2d5790eb396efcd7f90dea76a127cf15
-
由 Yanqin Jin 提交于
Summary: Currently, a few ldb commands do not check the execution result of database operations. This PR checks the execution results and tries to improve the error reporting. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8072 Test Plan: ``` make check ``` and ``` ASSERT_STATUS_CHECKED=1 make -j20 ldb python tools/ldb_test.py ``` Reviewed By: zhichao-cao Differential Revision: D27152466 Pulled By: riversand963 fbshipit-source-id: b94220496a4b3591b61c1d350f665860a6579f30
-