- 10 2月, 2023 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: Example failure: ``` [ RUN ] DBWriteTestInstance/DBWriteTest.LockWALInEffect/1 db/db_write_test.cc:646: Failure Put("key3", "value") Corruption: Not active ``` Presumably from a background compaction prior to Put. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11209 Test Plan: watch CI Reviewed By: akankshamahajan15 Differential Revision: D43147727 Pulled By: pdillinger fbshipit-source-id: a1c34ac5ab124bfe2f23205a30777990056e9082
-
- 09 2月, 2023 3 次提交
-
-
由 Peter Dillinger 提交于
Summary: In anticipation of using this to represent sets of CacheEntryRole for including or excluding kinds of blocks in block cache tiers, add significant new features to SmallEnumSet, including at least: * List initialization * Applicative constexpr operations * copy/move/equality ops * begin/end/const_iterator for iteration * Better comments Pull Request resolved: https://github.com/facebook/rocksdb/pull/11178 Test Plan: unit tests added/expanded Reviewed By: ltamasi Differential Revision: D42973723 Pulled By: pdillinger fbshipit-source-id: 40783486feda931c3f7c6fcc9a300acd6a4b0a0a
-
由 Peter Dillinger 提交于
Summary: We've seen many instances of build-linux-static_lib-alt_namespace-status_checked failing like this: ``` g++: fatal error: Killed signal terminated program cc1plus compilation terminated. make: *** [Makefile:2507: utilities/transactions/transaction_test.o] Error 1 ``` It's understandable that so many static linking jobs could exhaust memory. The executor only has 16 vcores, so going from 32 down to 24 shouldn't hurt build time. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11206 Test Plan: will watch CI Reviewed By: ajkr Differential Revision: D43137246 Pulled By: pdillinger fbshipit-source-id: 050b0f700c285dd913bcae8b4a76a44d04bb0356
-
由 anand76 提交于
Summary: Fix a bug in the calculation of the input buffer address/offset in log_reader.cc. The bug is when consecutive fragments of a compressed record are located at the same offset in the log reader buffer, the second fragment input buffer is treated as a leftover from the previous input buffer. As a result, the offset in the `ZSTD_inBuffer` is not reset. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11198 Test Plan: Add a unit test in log_test.cc that fails without the fix and passes with it. Reviewed By: ajkr, cbi42 Differential Revision: D43102692 Pulled By: anand1976 fbshipit-source-id: aa2648f4802c33991b76a3233c5a58d4cc9e77fd
-
- 08 2月, 2023 4 次提交
-
-
由 Levi Tamasi 提交于
Summary: The patch adds compaction filter support for wide-column entities by introducing a new `CompactionFilter` API called `FilterV3`. This API is called for regular key-values, merge operands, and wide-column entities as well. It is passed the existing value/operand or wide-column structure and it can update the value or columns or keep/delete/etc. the key-value as usual. For compatibility, the default implementation of `FilterV3` keeps all wide-column entities and falls back to calling `FilterV2` for plain old key-values and merge operands. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11196 Test Plan: `make check` Reviewed By: akankshamahajan15 Differential Revision: D43094147 Pulled By: ltamasi fbshipit-source-id: 75acabe9a35254f7f404ba6173ee9c2774382ebd
-
由 Hui Xiao 提交于
Summary: **Context/Summary:** As instructed by convenience.h comments, a few deprecated APIs are removed. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11120 Test Plan: - make check & CI - eyeball check on test semantics. Reviewed By: pdillinger Differential Revision: D42937507 Pulled By: hx235 fbshipit-source-id: a9e4709387da01b1d0e9148c2e210f02e9746ee1
-
由 Peter Dillinger 提交于
Summary: Continuous performance testing indicates there's a small performance hit with shared library (-fPIC) builds, so while retaining the motivation for https://github.com/facebook/rocksdb/issues/11168, we set the default for DEBUG_LEVEL=0 Makefile builds back to LIB_MODE=static. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11195 Test Plan: CI, with some updated checks and removal of some now obsolete LIB_MODE overrides Reviewed By: cbi42 Differential Revision: D43090576 Pulled By: pdillinger fbshipit-source-id: 755fe5d07005f85caf24e16f90228ffd46a6e250
-
由 Symious 提交于
Summary: Currently the option of "KForceOptimized" is not included in CompactRangeOptions.BottommostLevelCompaction. This PR is to add this option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11181 Reviewed By: ajkr Differential Revision: D43056453 Pulled By: cbi42 fbshipit-source-id: 22fd53f980ab1a86c61dd42e948902542065128f
-
- 07 2月, 2023 4 次提交
-
-
由 Peter Dillinger 提交于
Summary: Need to scp the .so files. Switched to tar+ssh to support symlinks, faster handling of multiple files, and compression. Also fixing some holes in 'make clean' as I've noticed files like 'librocksdb.so.7.7.0', 'librocksdb_test_debug.so', 'librocksdb_tools_debug.so' hanging around after `make clean` Pull Request resolved: https://github.com/facebook/rocksdb/pull/11194 Test Plan: Manually triggered regression test runs with change, manual `make clean` https://fburl.com/sandcastle/gnxy5lvc https://fburl.com/sandcastle/4pxodwh7 Reviewed By: cbi42 Differential Revision: D43069065 Pulled By: pdillinger fbshipit-source-id: 48552b5980956784a1fdb40638d9e8ad6db51900
-
由 Hui Xiao 提交于
Summary: **Context/Summary**: Simplify `TEST_F(DBWALTest, FixSyncWalOnObseletedWalWithNewManifestCausingMissingWAL)` based on https://github.com/facebook/rocksdb/pull/11016#pullrequestreview-1205020134 and delete unused sync points. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11186 Test Plan: - UT failed before fix in https://github.com/facebook/rocksdb/pull/10892 and passes after - Check UT not flaky when running with https://app.circleci.com/pipelines/github/facebook/rocksdb/21985/workflows/5f6cc355-78c1-46d8-89ee-0fd679725a8a/jobs/540878 Reviewed By: ajkr Differential Revision: D43034723 Pulled By: hx235 fbshipit-source-id: f503774987b8f3718505f99e95080a7fad28ac66
-
由 Peter Dillinger 提交于
Summary: There are a set of jobs using libbenchmark that have linker failures with new default LIB_MODE=shared. This change adds build-linux-run-microbench to the set using LIB_MODE=static to work around the linker failures. I haven't dug into how to fix them. There is another set of jobs using folly that have linker failures with new default LIB_MODE=shared. I tried fixing these by adding --shared-libs to the folly build, but that doesn't work. It kinda looks like the folly shared libs build is simply broken with the boost dependency: ``` /usr/bin/ld: /tmp/fbcode_builder_getdeps-ZrootZprojectZthird-partyZfollyZbuildZfbcode_builder-root/installed/boost-Z1Z72zV-c0-0f3HkylpzONnr1dsHYDaR2GyTLzYdkck/lib/libboost_filesystem.a(exception.o): relocation R_X86_64_PC32 against symbol `_ZTVN5boost10filesystem16filesystem_errorE' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status ``` I tried updating folly to the latest commit and that didn't help. Otherwise, I didn't dig deeper into fixing that so have added build-linux-clang-13-asan-ubsan-with-folly to the set using LIB_MODE=static Also since I saw a flaky failure (not the first time), increased the timeout on build-linux-unity-and-headers job. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11193 Test Plan: CI Reviewed By: cbi42 Differential Revision: D43061203 Pulled By: pdillinger fbshipit-source-id: c641671f93087f0214ea261ea895bccf657cb1a9
-
由 Alan Paxton 提交于
Summary: We had miscalculated (not sure if I suddenly can’t count, or if there is something else going on), and need to leave more overhead to get the benchmarks to run reliably under 1 hour. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11189 Reviewed By: cbi42 Differential Revision: D43052045 Pulled By: ajkr fbshipit-source-id: 3fe68432ed76a1f87d34129b0246e6b6a70a49f2
-
- 04 2月, 2023 5 次提交
-
-
由 Peter Dillinger 提交于
Summary: Seen only in post-PR CI job benchmark-linux. Some context: https://stackoverflow.com/questions/30680550/c-gettid-was-not-declared-in-this-scope Pull Request resolved: https://github.com/facebook/rocksdb/pull/11184 Test Plan: watch CI Reviewed By: cbi42 Differential Revision: D43013891 Pulled By: pdillinger fbshipit-source-id: 48b3b7231080a0f803fdc36d13946d5524770302
-
由 Peter Dillinger 提交于
Summary: With https://github.com/facebook/rocksdb/issues/11150 this becomes a practical change that I think is overall good for developer efficiency. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11168 Test Plan: More efficient build of all unit tests and tools: ``` $ git clean -fdx $ du -sh . 522M . $ /usr/bin/time make -j32 LIB_MODE=static ... 14270.63user 1043.33system 11:19.85elapsed 2252%CPU (0avgtext+0avgdata 1929944maxresident)k ... $ du -sh . 62G . $ ``` Vs. ``` $ git clean -fdx $ du -sh . 522M . $ /usr/bin/time make -j32 LIB_MODE=shared ... 9479.87user 478.26system 7:20.82elapsed 2258%CPU (0avgtext+0avgdata 1929272maxresident)k ... $ du -sh . 5.4G . $ ``` So 1/3 less build time and >90% less space usage. Individual unit test edit-compile-run is not too different. Modifying an average unit test source file: ``` $ touch db/version_builder_test.cc $ /usr/bin/time make -j32 LIB_MODE=static version_builder_test ... 34.74user 3.37system 0:38.29elapsed 99%CPU (0avgtext+0avgdata 945520maxresident)k ``` Vs. ``` $ touch db/version_builder_test.cc $ /usr/bin/time make -j32 LIB_MODE=shared version_builder_test ... 116.26user 43.91system 0:28.65elapsed 559%CPU (0avgtext+0avgdata 675160maxresident)k ``` A little faster with shared. However, modifying an average DB implementation file has an extra linking step with shared lib: ``` $ touch db/db_impl/db_impl_files.cc $ /usr/bin/time make -j32 LIB_MODE=static version_builder_test ... 33.17user 5.13system 0:39.70elapsed 96%CPU (0avgtext+0avgdata 945544maxresident)k ``` Vs. ``` $ touch db/db_impl/db_impl_files.cc $ /usr/bin/time make -j32 LIB_MODE=shared version_builder_test ... 40.80user 4.66system 0:45.54elapsed 99%CPU (0avgtext+0avgdata 1056340maxresident)k ``` A little slower with shared. On the whole, should be faster and lighter weight because of the many unit test files case Reviewed By: cbi42 Differential Revision: D42894004 Pulled By: pdillinger fbshipit-source-id: 9e827e52ace79b86f849b6a24466e318b4b605a7
-
由 Peter Dillinger 提交于
Summary: LIB_MODE=shared is much more efficient for building all the unit tests but comes with the downside of ugly stack traces, generally missing name demangling and source line info. Searching the internet suggests the reliable way to get stack traces with dynamic loading is with gdb. This change automatically tries to use gdb to get a stack trace if built with LIB_MODE=shared, and only on Linux because that's where we have the capability to attach to the proper thread. (We could revise the exact conditions in the future.) If there's a failure invoking gdb, it falls back on the old method. Obscure details of making the output reasonable / pretty are in the source code comments. Based on this, it was easy to make it so that running a test command with ROCKSDB_DEBUG=1 would invoke gdb whenever the stack trace handler was invoked, so I included that. Intended follow-up: make LIB_MODE=shared the new default `make` build config Pull Request resolved: https://github.com/facebook/rocksdb/pull/11150 Test Plan: manual, mostly by injecting an "assert(false)" into a unit test and trying different build modes etc. Although gdb is slower to start showing stack trace output, it seems overall faster in many if not most cases, presumably because it doesn't reload the symbol table for each stack entry. At least with parallel test runs, having many tests dumping stacks with the old method can take so long it appears to hang the test run. Reviewed By: cbi42 Differential Revision: D42894064 Pulled By: pdillinger fbshipit-source-id: 608143309d8c69c40049c9a4abcde4f22e87b4d8
-
由 Peter Dillinger 提交于
Summary: This option has long been intended to be set to false by default and deprecated. It might never be practical to completely remove the feature, so that we can continue to test for backward compatibility by keeping the ability to generate DBs in the old way. Also improved API comments. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11179 Test Plan: existing tests (with one tiny update) Reviewed By: hx235 Differential Revision: D42973927 Pulled By: pdillinger fbshipit-source-id: e9bc161cb933266e094aea2dff8cc03753c39dab
-
由 Peter Dillinger 提交于
Summary: Fixes https://github.com/facebook/rocksdb/issues/11160 By counting the number of stalls placed on a write queue, we can check in UnlockWAL() whether the stall present at the start of UnlockWAL() has been cleared by the end, or wait until it's cleared. More details in code comments and new unit test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11172 Test Plan: unit test added. Yes, it uses sleep to amplify failure on buggy behavior if present, but using a sync point to only allow new behavior would fail with the old code only because it doesn't contain the new sync point. Basically, using a sync point in UnlockWAL() could easily mask a regression by artificially limiting key behaviors. The test would only check that UnlockWAL() invokes code that *should* do the right thing, without checking that it *does* the right thing. Reviewed By: ajkr Differential Revision: D42894341 Pulled By: pdillinger fbshipit-source-id: 15c9da0ca383e6aec845b29f5447d76cecbf46c3
-
- 03 2月, 2023 4 次提交
-
-
由 anand76 提交于
Summary: Currently, we incorrectly return a Status::Corruption to the MultiGet caller if the file system ReadAsync cannot issue a read and returns an error for some reason, such as IOStatus::NotSupported(). In this PR, we copy the ReadAsync error to the request status so it can be returned to the user. Tests: Update existing unit tests and add a new one for this scenario Pull Request resolved: https://github.com/facebook/rocksdb/pull/11171 Reviewed By: akankshamahajan15 Differential Revision: D42950057 Pulled By: anand1976 fbshipit-source-id: 85ffcb015fa6c064c311f8a28488fec78c487869
-
由 Yu Zhang 提交于
Summary: Enable the set of crash test for when user defined timestamp is enabled in combination with BlobDB. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11163 Test Plan: `make check` and `db_stress`/`db_crashtest.py` with various combinations. Reviewed By: ltamasi Differential Revision: D42906457 Pulled By: jowlyzhang fbshipit-source-id: 6bec6449a4213b536c787420ff30a7d17b676deb
-
由 changyubi 提交于
Summary: benchmark-linux is failing on main branch after https://github.com/facebook/rocksdb/issues/11074 with the following error msg: ``` /usr/bin/time -f '%e %U %S' -o /tmp/benchmark-results/8.0.0/benchmark_overwriteandwait.t1.s0.log.time numactl --interleave=all timeout 1200 ./db_bench --benchmarks=overwrite,waitforcompaction,stats --use_existing_db=1 --sync=0 --level0_file_num_compaction_trigger=4 --level0_slowdown_writes_trigger=20 --level0_stop_writes_trigger=30 --max_background_jobs=4 --max_write_buffer_number=8 --undefok=use_blob_cache,use_shared_block_and_blob_cache,blob_cache_size,blob_cache_numshardbits,prepopulate_blob_cache,multiread_batched,cache_low_pri_pool_ratio,prepopulate_block_cache --db=/tmp/rocksdb-benchmark-datadir --wal_dir=/tmp/rocksdb-benchmark-datadir --num=20000000 --key_size=20 --value_size=400 --block_size=8192 --cache_size=10737418240 --cache_numshardbits=6 --compression_max_dict_bytes=0 --compression_ratio=0.5 --compression_type=none --bytes_per_sync=1048576 --cache_index_and_filter_blocks=1 --cache_high_pri_pool_ratio=0.5 --cache_low_pri_pool_ratio=0 --benchmark_write_rate_limit=0 --write_buffer_size=16777216 --target_file_size_base=16777216 --max_bytes_for_level_base=67108864 --verify_checksum=1 --delete_obsolete_files_period_micros=62914560 --max_bytes_for_level_multiplier=8 --statistics=0 --stats_per_interval=1 --stats_interval_seconds=20 --report_interval_seconds=1 --histogram=1 --memtablerep=skip_list --bloom_bits=10 --open_files=-1 --subcompactions=1 --compaction_style=0 --num_levels=8 --min_level_to_compress=3 --level_compaction_dynamic_level_bytes=true --pin_l0_filter_and_index_blocks_in_cache=1 --duration=600 --threads=1 --merge_operator="put" --seed=1675372532 --report_file=/tmp/benchmark-results/8.0.0/benchmark_overwriteandwait.t1.s0.log.r.csv 2>&1 | tee -a /tmp/benchmark-results/8.0.0/benchmark_overwriteandwait.t1.s0.log /usr/bin/time: cannot run numactl: No such file or directory ``` This PR removes the newly added NUMA setting. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11180 Test Plan: check next main branch run for benchmark-linux Reviewed By: ajkr Differential Revision: D42975930 Pulled By: cbi42 fbshipit-source-id: f084d39aeba9877c0752502e879c5e612b507653
-
由 Alan Paxton 提交于
Summary: First, we made a small reduction in DURATION_RW as runs were exceeding 1 hour and colliding with subsequent runs. See Mark Callaghan’s blog post at http://smalldatum.blogspot.com/2023/01/variance-in-rocksdb-benchmarks-on-cloud.html Configuration parameters which are not consistent with the following email from Mark (see the blog post for more context) have been updated. Where Mark has defined the parameter and we haven't, we define it explicitly. We will need to further monitor for an expected reduction in variance of test times: To match what I did: --- nsecs=1800 dbdir=/data/m/rx resultdir=bm.lc.nt1.cm1.d0 env WRITE_BUFFER_SIZE_MB=16 TARGET_FILE_SIZE_BASE_MB=16 MAX_BYTES_FOR_LEVEL_BASE_MB=64 MAX_BACKGROUND_JOBS=4 NUM_KEYS=20000000 CACHE_SIZE_MB=10240 DURATION_RW=$nsecs DURATION_RO=$nsecs MB_WRITE_PER_SEC=2 NUM_THREADS=1 COMPRESSION_TYPE=none CACHE_INDEX_AND_FILTER_BLOCKS=1 VALUE_SIZE=400 NUMA=1 MIN_LEVEL_TO_COMPRESS=3 COMPACTION_STYLE=leveled bash benchmark_compare.sh $dbdir $resultdir 7.8.fb env WRITE_BUFFER_SIZE_MB=16 TARGET_FILE_SIZE_BASE_MB=16 MAX_BYTES_FOR_LEVEL_BASE_MB=64 MAX_BACKGROUND_JOBS=4 NUM_KEYS=200000000 CACHE_SIZE_MB=10240 DURATION_RW=$nsecs DURATION_RO=$nsecs MB_WRITE_PER_SEC=2 NUM_THREADS=1 COMPRESSION_TYPE=lz4 CACHE_INDEX_AND_FILTER_BLOCKS=1 VALUE_SIZE=400 NUMA=1 MIN_LEVEL_TO_COMPRESS=3 COMPACTION_STYLE=leveled bash benchmark_compare.sh $dbdir $resultdir 7.8.fb Pull Request resolved: https://github.com/facebook/rocksdb/pull/11074 Reviewed By: ajkr Differential Revision: D42969668 Pulled By: cbi42 fbshipit-source-id: 1ea4e6a3901be4016108f93817eb58f74baac21a
-
- 02 2月, 2023 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/11177 Reviewed By: cbi42 Differential Revision: D42948946 Pulled By: ajkr fbshipit-source-id: 783d3d9007faaa036923a0364cdd0bfbd8e78062
-
由 Levi Tamasi 提交于
Summary: The patch makes some code quality enhancements in `CompactionIterator::InvokeFilterIfNeeded` including the renaming of `filter` (which is most likely a remnant of the days before the `FilterV2` API when the compaction filter used to return a boolean) to `decision`, the removal of some outdated comments, the elimination of an `error` flag which was only used in one failure case out of many, as well as some small stylistic improvements. (Some the above will also come in handy when adding compaction filter support for wide-column entities.) Pull Request resolved: https://github.com/facebook/rocksdb/pull/11174 Test Plan: `make check` Reviewed By: akankshamahajan15 Differential Revision: D42901408 Pulled By: ltamasi fbshipit-source-id: ab382d59a4990c5dfe1cee219d49e1d80902b666
-
- 01 2月, 2023 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This PR adds logic to the `RunManualCompaction()` loop to check for cancellation before waiting on any conflicting compactions to finish. In case of cancellation, `RunManualCompaction()` no longer waits on conflicting compactions Pull Request resolved: https://github.com/facebook/rocksdb/pull/11165 Test Plan: repro test case Reviewed By: cbi42 Differential Revision: D42864058 Pulled By: ajkr fbshipit-source-id: ea4dd1a8f294abe212905495a8fbe8f07fca3f5a
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/11144 Test Plan: Ran a simple blackbox crash test. Reviewed By: akankshamahajan15 Differential Revision: D42791464 Pulled By: ltamasi fbshipit-source-id: 8eb6e62f0bc47f709816136ff3ded0a41d04fab8
-
由 Levi Tamasi 提交于
Summary: The patch fixes a feature interaction bug between BlobDB and the `GetEntity` API: without the patch, `GetEntity` would return the blob reference (wrapped into a single-column entity) instead of the actual blob value. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11162 Test Plan: `make check` Reviewed By: akankshamahajan15 Differential Revision: D42854092 Pulled By: ltamasi fbshipit-source-id: f750d0ff57def107da16f545077ddce9860ff21a
-
- 31 1月, 2023 3 次提交
-
-
由 Peter Dillinger 提交于
Summary: The previous API comments for LockWAL didn't provide much about why you might want to use it, and didn't really meet what one would infer its contract was. Also, LockWAL was not in db_stress / crash test. In this change: * Implement a counting semantics for LockWAL()+UnlockWAL(), so that they can safely be used concurrently across threads or recursively within a thread. This should make the API much less bug-prone and easier to use. * Make sure no UnlockWAL() is needed after non-OK LockWAL() (to match RocksDB conventions) * Make UnlockWAL() reliably return non-OK when there's no matching LockWAL() (for debug-ability) * Clarify API comments on LockWAL(), UnlockWAL(), FlushWAL(), and SyncWAL(). Their exact meanings are not obvious, and I don't think it's appropriate to talk about implementation mutexes in the API comments, but about what operations might block each other. * Add LockWAL()/UnlockWAL() to db_stress and crash test, mostly to check for assertion failures, but also checks that latest seqno doesn't change while WAL is locked. This is simpler to add when LockWAL() is allowed in multiple threads. * Remove unnecessary use of sync points in test DBWALTest::LockWal. There was a bug during development of above changes that caused this test to fail sporadically, with and without this sync point change. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11143 Test Plan: unit tests added / updated, added to stress/crash test Reviewed By: ajkr Differential Revision: D42848627 Pulled By: pdillinger fbshipit-source-id: 6d976c51791941a31fd8fbf28b0f82e888d9f4b4
-
由 sdong 提交于
Summary: Seeting this error in stress test: db_stress: internal_repo_rocksdb/repo/db_stress_tool/db_stress_test_base.cc:2459: void rocksdb::StressTest::Open(rocksdb::SharedState *): Assertion `txn_db_ == nullptr' failed. Received signal 6 (Aborted) ...... It doesn't appear that txn_db_ is set to nullptr at all. We set ithere. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11164 Test Plan: Run db_stress transaction and non-transation with low kill rate and see restarting without assertion Reviewed By: ajkr Differential Revision: D42855662 fbshipit-source-id: 06816d37cce9c94a81cb54ab238fb73aa102ed46
-
由 Yu Zhang 提交于
Summary: Use the user key on sst file for blob verification for `Get` and `MultiGet` instead of the user key passed from caller. Add tests for `Get` and `MultiGet` operations when user defined timestamp feature is enabled in a BlobDB. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11105 Test Plan: make V=1 db_blob_basic_test ./db_blob_basic_test --gtest_filter="DBBlobTestWithTimestamp.*" Reviewed By: ltamasi Differential Revision: D42716487 Pulled By: jowlyzhang fbshipit-source-id: 5987ecbb7e56ddf46d2467a3649369390789506a
-
- 28 1月, 2023 3 次提交
-
-
由 akankshamahajan 提交于
Summary: Migrate ExternalSSTTestEnv to FileSystemWrapper Pull Request resolved: https://github.com/facebook/rocksdb/pull/11139 Reviewed By: anand1976 Differential Revision: D42780180 Pulled By: akankshamahajan15 fbshipit-source-id: 9a4448c9fe5186b518235fe11e1a34dcad897cdd
-
由 sdong 提交于
Summary: We haven't been actively mantaining RocksDB LITE recently and the size must have been gone up significantly. We are removing the support. Most of changes were done through following comments: unifdef -m -UROCKSDB_LITE `git grep -l ROCKSDB_LITE | egrep '[.](cc|h)'` by Peter Dillinger. Others changes were manually applied to build scripts, CircleCI manifests, ROCKSDB_LITE is used in an expression and file db_stress_test_base.cc. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11147 Test Plan: See CI Reviewed By: pdillinger Differential Revision: D42796341 fbshipit-source-id: 4920e15fc2060c2cd2221330a6d0e5e65d4b7fe2
-
由 Yu Zhang 提交于
Summary: Remove the util functions in options_util.h that have previously been marked deprecated. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11126 Test Plan: `make check` Reviewed By: ltamasi Differential Revision: D42757496 Pulled By: jowlyzhang fbshipit-source-id: 2a138a3c207d0e0e0bbb4d99548cf2cadb44bcfb
-
- 27 1月, 2023 6 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/11133 Test Plan: - ran it a few times on a mismatching DB+expected state; verified error messages look right: ``` Verification failed for column family 0 key 000000000000D553000000000000014C0000000000000142 (163988): value_from_db: , value_from_expected: 25E7B53421202322, msg: GetMergeOperands verification: Value not found: NotFound: Verification failed for column family 0 key 000000000000AAE2787878 (131123): value_from_db: , value_from_expected: B2A69C18B6B7B4B5BABBB8B9BEBFBCBDA2A3A0A1A6A7A4A5, msg: Iterator verification: Value not found: NotFound: Verification failed for column family 0 key 00000000000080C6000000000000004C78787878 (98409): value_from_db: , value_from_expected: 67AB7E1E636261606F6E6D6C6B6A6968, msg: Get verification: Value not found: NotFound: ``` Reviewed By: hx235 Differential Revision: D42757072 Pulled By: ajkr fbshipit-source-id: b0a4a0aaa5be5d110434324853ac92aaa6972d89
-
由 Changyu Bi 提交于
Summary: user should use FileSystem::CreateFromString() instead. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11122 Reviewed By: ajkr Differential Revision: D42727580 Pulled By: cbi42 fbshipit-source-id: c68b17bb82ba9dee46ba23b677d87ecf0a1e06c8
-
由 Karim TAAM 提交于
Summary: ## Description In this issue https://github.com/facebook/rocksdb/issues/11002 we found that when we use rocksdb with the `verify checksum` read_option to false the verification is done anyway By analyzing the code along the stacktrace I saw that at the level of https://github.com/facebook/rocksdb/compare/main...matkt:feature/use-verify-checksum-in-block-based-table-reader?expand=1#diff-57ed8c49db2bdd4db7618646a177397674bbf25beacacecb104070071d30129f we are not keeping all the options and we forget the `verify_checksum` the comment in this class suggests that it should be managed https://github.com/facebook/rocksdb/compare/main...matkt:feature/use-verify-checksum-in-block-based-table-reader?expand=1#diff-57ed8c49db2bdd4db7618646a177397674bbf25beacacecb104070071d30129fL581 <img width="1724" alt="204511641-86ab4b9b-45e5-4a2b-a13d-81fa26435d38" src="https://user-images.githubusercontent.com/26581503/213152802-c46bc1c7-a3a2-4a6f-9bb1-bf92ee93af7a.png"> this PR just adds the line to manage the `verify checksum` ## Tests - Running unit tests - Test without setting `verify checksum` and verifying that we are calling the checksum code - Test by setting `verify checksum` to true and verifying that we are calling the checksum code - Test by setting `verify checksum` to false and verifying that we are **not** calling the checksum code Pull Request resolved: https://github.com/facebook/rocksdb/pull/11099 Reviewed By: cbi42 Differential Revision: D42679881 Pulled By: ajkr fbshipit-source-id: c7dd10768282fd0699f7e1bf397ceb7adbea4ab6
-
由 Andrew Kryczka 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/11136 Test Plan: the provided unit test used to fail due to `GetMergeOperands()` returning `Status::MergeInProgress()`; it passes now because the `GetMergeOperands()` call returns `Status::OK()` Reviewed By: pdillinger Differential Revision: D42759198 Pulled By: ajkr fbshipit-source-id: 878f9f40ccc1d7e2fe7b1352814bae3a49c19939
-
由 dependabot[bot] 提交于
Summary: Bumps [commonmarker](https://github.com/gjtorikian/commonmarker) from 0.23.6 to 0.23.7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/gjtorikian/commonmarker/releases">commonmarker's releases</a>.</em></p> <blockquote> <h2>v0.23.7</h2> <h2>What's Changed</h2> <ul> <li>C API stable test by <a href="https://github.com/gjtorikian"><code>@gjtorikian</code></a> in <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/201">gjtorikian/commonmarker#201</a></li> <li>Update to 29.0.gfm.7 by <a href="https://github.com/anticomputer"><code>@anticomputer</code></a> in <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/224">gjtorikian/commonmarker#224</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/gjtorikian/commonmarker/compare/v0.23.6...v0.23.7">https://github.com/gjtorikian/commonmarker/compare/v0.23.6...v0.23.7</a></p> <h2>v0.23.7.pre1</h2> <h2>What's Changed</h2> <ul> <li>C API stable test by <a href="https://github.com/gjtorikian"><code>@gjtorikian</code></a> in <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/201">gjtorikian/commonmarker#201</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/gjtorikian/commonmarker/compare/v0.23.6...v0.23.7.pre1">https://github.com/gjtorikian/commonmarker/compare/v0.23.6...v0.23.7.pre1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/gjtorikian/commonmarker/blob/main/CHANGELOG.md">commonmarker's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <h2><a href="https://github.com/gjtorikian/commonmarker/tree/v1.0.0.pre6">v1.0.0.pre6</a> (2023-01-09)</h2> <p><a href="https://github.com/gjtorikian/commonmarker/compare/v1.0.0.pre5...v1.0.0.pre6">Full Changelog</a></p> <p><strong>Closed issues:</strong></p> <ul> <li>Cargo.lock prevents Ruby 3.2.0 from installing commonmarker v1.0.0.pre4 <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/issues/211">https://github.com/facebook/rocksdb/issues/211</a></li> </ul> <p><strong>Merged pull requests:</strong></p> <ul> <li>always use rb_sys (don't use Ruby's emerging cargo tooling where available) <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/213">https://github.com/facebook/rocksdb/issues/213</a> (<a href="https://github.com/kivikakk">kivikakk</a>)</li> </ul> <h2><a href="https://github.com/gjtorikian/commonmarker/tree/v1.0.0.pre5">v1.0.0.pre5</a> (2023-01-08)</h2> <p><a href="https://github.com/gjtorikian/commonmarker/compare/v1.0.0.pre4...v1.0.0.pre5">Full Changelog</a></p> <p><strong>Merged pull requests:</strong></p> <ul> <li>Provide 3.2 build support <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/212">https://github.com/facebook/rocksdb/issues/212</a> (<a href="https://github.com/gjtorikian">gjtorikian</a>)</li> </ul> <h2><a href="https://github.com/gjtorikian/commonmarker/tree/v1.0.0.pre4">v1.0.0.pre4</a> (2022-12-28)</h2> <p><a href="https://github.com/gjtorikian/commonmarker/compare/v1.0.0.pre3...v1.0.0.pre4">Full Changelog</a></p> <p><strong>Closed issues:</strong></p> <ul> <li>Will the cmark-gfm branch continue to be maintained for awhile? <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/issues/207">https://github.com/facebook/rocksdb/issues/207</a></li> </ul> <p><strong>Merged pull requests:</strong></p> <ul> <li>Implement native syntax highlighting <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/209">https://github.com/facebook/rocksdb/issues/209</a> (<a href="https://github.com/gjtorikian">gjtorikian</a>)</li> <li>Bump magnus from 0.4.3 to 0.4.4 <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/208">https://github.com/facebook/rocksdb/issues/208</a> (<a href="https://github.com/apps/dependabot">dependabot[bot]</a>)</li> <li>Bump magnus from 0.4.2 to 0.4.3 <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/206">https://github.com/facebook/rocksdb/issues/206</a> (<a href="https://github.com/apps/dependabot">dependabot[bot]</a>)</li> <li>Bump comrak from 0.14.0 to 0.15.0 <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/205">https://github.com/facebook/rocksdb/issues/205</a> (<a href="https://github.com/apps/dependabot">dependabot[bot]</a>)</li> <li>Bump magnus from 0.4.1 to 0.4.2 <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/204">https://github.com/facebook/rocksdb/issues/204</a> (<a href="https://github.com/apps/dependabot">dependabot[bot]</a>)</li> </ul> <h2><a href="https://github.com/gjtorikian/commonmarker/tree/v1.0.0.pre3">v1.0.0.pre3</a> (2022-11-30)</h2> <p><a href="https://github.com/gjtorikian/commonmarker/compare/v1.0.0.pre.2...v1.0.0.pre3">Full Changelog</a></p> <p><strong>Closed issues:</strong></p> <ul> <li>Code block incorrectly parsed in commonmarker 1.0.0.pre <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/issues/202">https://github.com/facebook/rocksdb/issues/202</a></li> </ul> <p><strong>Merged pull requests:</strong></p> <ul> <li>Windows build <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/pull/197">https://github.com/facebook/rocksdb/issues/197</a> (<a href="https://github.com/gjtorikian">gjtorikian</a>)</li> </ul> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/gjtorikian/commonmarker/commit/734fd86c9719643e0ef058d983daeaf2523ea78b"><code>734fd86</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/issues/224">https://github.com/facebook/rocksdb/issues/224</a> from gjtorikian/update-to-29.0.gfm.7</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/2e724ec52a5f66116c03682b6cc76c659f8efaf5"><code>2e724ec</code></a> Turned off Rubocop.</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/9c923b0bfdb2e4087779042a483a2d6957dceb7a"><code>9c923b0</code></a>
💎 release 0.23.7</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/30419c25e8566f3e02a0fad76f5e1a82341edeb1"><code>30419c2</code></a> Added call to cmark_init_standard_node_flags()</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/9007c3798f2f6fcff80e49a7ced317887f259326"><code>9007c37</code></a> Update cmark-upstream to <a href="https://github.com/github/cmark-gfm/commit/57d5e093e">https://github.com/github/cmark-gfm/commit/57d5e093e</a>...</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/1cfec133730ecdf683b90528a6bb7253b68a8208"><code>1cfec13</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/gjtorikian/commonmarker/issues/201">https://github.com/facebook/rocksdb/issues/201</a> from gjtorikian/c-api-stable-test</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/bbf631b413cf012490ccd63d423e69dfa855369b"><code>bbf631b</code></a> lint</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/5b807a115d02dc7456467dbed871a3e91733ca7b"><code>5b807a1</code></a> ease up</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/9a24e6d2fe3a1d384e29702c1a223a4fea5bdc87"><code>9a24e6d</code></a> Test fake version</li> <li><a href="https://github.com/gjtorikian/commonmarker/commit/d8a43bc73a6ff5c15cd12ce7dcf49902ee8031ec"><code>d8a43bc</code></a> Allow for manual dispatch</li> <li>Additional commits viewable in <a href="https://github.com/gjtorikian/commonmarker/compare/v0.23.6...v0.23.7">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=commonmarker&package-manager=bundler&previous-version=0.23.6&new-version=0.23.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebook/rocksdb/network/alerts). </details> Pull Request resolved: https://github.com/facebook/rocksdb/pull/11128 Reviewed By: ajkr Differential Revision: D42752086 Pulled By: cbi42 fbshipit-source-id: 6992b6f1096400a6b10b79fe36bf955fec841b71 -
由 Levi Tamasi 提交于
Summary: Some histograms were incorrectly categorized as tickers. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11140 Reviewed By: anand1976 Differential Revision: D42780030 Pulled By: ltamasi fbshipit-source-id: 5aca8ec5baad8f73676aaa9d6cdbbd2a619c8a89
-
- 26 1月, 2023 2 次提交
-
-
由 akankshamahajan 提交于
Summary: Migrate derived classes from EnvWrapper to FileSystemWrapper so we can eventually deprecate the storage methods in Env. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11125 Test Plan: CircleCI jobs Reviewed By: anand1976 Differential Revision: D42732241 Pulled By: akankshamahajan15 fbshipit-source-id: c89a70a79fcfb13e158bf8919b1a87a9de133222
-
由 sdong 提交于
Summary: Since compressed block cache is removed, those stats are not needed. They are removed in different PR in case there is a problem with it. The stats are removed in the same way in https://github.com/facebook/rocksdb/pull/11131/ . HISTORY.md was already updated by mistake, and it would be correct after merging this PR. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11135 Test Plan: Watch CI Reviewed By: ltamasi Differential Revision: D42757616 fbshipit-source-id: bd7cb782585c8535ce5784295225c376f3011f35
-