- 03 1月, 2020 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: allow_concurrent_memtable_write is incompatible with non-zero max_successive_merges. Although we check this at runtime, we currently don't prevent the user from setting this combination in options. This has led to stress tests to fail with this combination is tried in ::SetOptions. The patch fixes that. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6254 Differential Revision: D19265819 Pulled By: maysamyabandeh fbshipit-source-id: 47f2e2dc26fe0972c7152f4da15dadb9703f1179
-
- 31 12月, 2019 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: Was probably a false signal suggesting a problem in https://github.com/facebook/rocksdb/issues/6217 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6251 Test Plan: 'make crash_test' Differential Revision: D19246951 Pulled By: pdillinger fbshipit-source-id: 3e4fafcd9a7cf5f19ffd207b90279ba615145d6f
-
- 27 12月, 2019 1 次提交
-
-
由 kim.sanghyun 提交于
Summary: meareTime() -> measureTime() Pull Request resolved: https://github.com/facebook/rocksdb/pull/6248 Differential Revision: D19231406 Pulled By: riversand963 fbshipit-source-id: 20de4a43a5478b4a3e7853e1fb70b09ccadbf985
-
- 25 12月, 2019 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: Clang analyzer was falsely reporting on use of txn=nullptr. Added a new const variable so that it can properly prune impossible control flows. Also, 'make analyze' previously required setting USE_CLANG=1 as an environment variable, not a make variable, or else compilation errors like g++: error: unrecognized command line option ‘-Wshorten-64-to-32’ Now USE_CLANG is not required for 'make analyze' (it's implied) and you can do an incremental analysis (recompile what has changed) with 'USE_CLANG=1 make analyze_incremental' Pull Request resolved: https://github.com/facebook/rocksdb/pull/6244 Test Plan: 'make -j24 analyze', 'make crash_test' Differential Revision: D19225950 Pulled By: pdillinger fbshipit-source-id: 14f4039aa552228826a2de62b2671450e0fed3cb
-
- 24 12月, 2019 1 次提交
-
-
由 Peter Dillinger 提交于
Summary: This commit is suspected in some crash test failures such as Verification failed for column family 0 key 78438077: Value not found: NotFound: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6243 Test Plan: 'make check' and start 'make crash_test' Differential Revision: D19220495 Pulled By: pdillinger fbshipit-source-id: 6c4709cee80ab4344e06ce360f51e947d79fb3fa
-
- 22 12月, 2019 2 次提交
-
-
由 Peter Dillinger 提交于
Summary: Seeing crash test failures like db_stress: db_stress_tool/no_batched_ops_stress.cc:271: virtual rocksdb::Status rocksdb::NonBatchedOpsStressTest::TestPrefixScan(rocksdb::ThreadState*, const rocksdb::ReadOptions&, const std::vector<int>&, const std::vector<long int>&): Assertion `count <= GetPrefixKeyCount(prefix.ToString(), upper_bound)' failed. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6238 Differential Revision: D19210312 Pulled By: pdillinger fbshipit-source-id: 4d2c35c38f418b408e01c7ba22adf6983ae67d44
-
由 Peter Dillinger 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6237 Differential Revision: D19210304 Pulled By: pdillinger fbshipit-source-id: f6f050e995f4d210f812bb1d2020adbd751e1d5a
-
- 21 12月, 2019 14 次提交
-
-
由 anand76 提交于
Summary: Call Transaction::MultiGet from TestMultiGet() in db_stress. We add some Puts/Merges/Deletes into the transaction in order to exercise MultiGetFromBatchAndDB. There is no data verification on read, just check status. There is currently no read data verification in db_stress as it requires synchronization with writes. It needs to be tackled separately. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6227 Test Plan: make crash_test_with_txn Differential Revision: D19204611 Pulled By: anand1976 fbshipit-source-id: 770d0e30d002e88626c264c58103f1d709bb060c
-
由 sdong 提交于
Summary: Recently db_stress starts to use a special Env that keeps all manifest files. This should not apply to checkpoint directory and causes test failure like this: Verification failed: Checkpoint gave inconsistent state. Status is IO error: While mkdir: /dev/shm/rocksdb/rocksdb_crashtest_whitebox/.checkpoint27.tmp: File exists Pull Request resolved: https://github.com/facebook/rocksdb/pull/6233 Test Plan: Run crash_test with high chance of checkpoint and make sure it doesn't reproduce. Differential Revision: D19207250 fbshipit-source-id: 12a931379e2e0572bb84aa658b6d03770c8551d4
-
由 sdong 提交于
Summary: Listners are one source of bugs because we frequently release some mutex to invoke them, which introduce race conditions. Implement all callback functions in db_stress's listener class, and randomly sleep. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6197 Test Plan: Run crash_test for a while and see no obvious problem. Differential Revision: D19134015 fbshipit-source-id: b9ea8be9366e4501759119520cd4f204943538f6
-
由 sdong 提交于
Summary: db_stress to execute DB::GetApproximateSizes() with randomized keys and options. Return value is not validated but error will be reported. Two ways to generate the range keys: (1) two random keys; (2) a small range. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6213 Test Plan: (1) run "make crash_test" for a while; (2) hack the code to ingest some errors to see it is reported. Differential Revision: D19204665 fbshipit-source-id: 652db36f13bcb5a3bd8fe4a10c0aa22a77a0bce2
-
由 anand76 提交于
Summary: Currently, db_stress generates fixed length keys of 8 bytes. This patch adds the ability to generate variable length keys. Most of the db_stress code continues to work with a numeric key randomly generated, and the numeric key also acts as an index into the values_ array. The numeric key is mapped to a variable length string key in a deterministic way. Furthermore, the ordering is preserved. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6165 Test Plan: run make crash_test Differential Revision: D19204646 Pulled By: anand1976 fbshipit-source-id: d2d46a96615b4832a8be2a981f5913905f0e1ca7
-
由 sdong 提交于
Summary: Several improvements to crash_test/stress_test: (1) Stress_test to support an parameter of bottommost compression (2) Rename those FLAGS_* variables that are not gflags to avoid confusion (3) Crash_test to randomly generate compression type for bottommost compression with half the chance. (4) Stress_test to sanitize unsupported compression type to snappy, so that crash_test to cover all possible compression types and people don't need to worry about they don't support all comrpession types in their environment. (5) In crash_test, when generating db_stress command, sort arguments in alphabeta order, so that it is easier to find value for a specific argument. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6215 Test Plan: Run "make crash_test" for a while and see the botommost option shown in LOG files. Differential Revision: D19171255 fbshipit-source-id: d7001e246c4ff9ee5760776eea0be97738650735
-
由 sdong 提交于
Summary: 1. Cover SeekToFirst() and SeekToLast(). 2. Try to record the history of iterator operations. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6166 Test Plan: Do some manual changes in the code to cover the failure cases and see the error printing is correct and SeekToFirst() and SeekToLast() sometimes show up. Differential Revision: D19047079 fbshipit-source-id: 1ed616f919fe4d32c0a021fc37932a7bd3063bcd
-
由 Adam Retter 提交于
Summary: Closes https://github.com/facebook/rocksdb/issues/6183 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6186 Differential Revision: D19201281 Pulled By: pdillinger fbshipit-source-id: 1c96b4ea09e826f91e44b0009eba3de0991d9053
-
由 Adam Retter 提交于
Summary: There are no API changes ;-) Pull Request resolved: https://github.com/facebook/rocksdb/pull/6218 Differential Revision: D19200373 Pulled By: pdillinger fbshipit-source-id: 58d34b01ea53b75a1eccbd72f8b14d6256a7380f
-
由 Zhichao Cao 提交于
Summary: Add the verification in operateDB to verify GetLiveFiles, GetSortedWalFiles and GetCurrentWalFile. The test will be called every 1 out of N, N is decided by get_live_files_and_wal_files_one_i, whose default is 1000000. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6224 Test Plan: pass db_stress default run. Differential Revision: D19183358 Pulled By: zhichao-cao fbshipit-source-id: 20073cf72ede77a3e0d3cf5f28304f1f605d2b1a
-
由 Yanqin Jin 提交于
Summary: As title. We can run non-cf-consistency stress tests with verify_db_one_in>0, thus remove the check added previously. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6231 Test Plan: ``` make crash_test ``` Differential Revision: D19198295 Pulled By: riversand963 fbshipit-source-id: e874c701bb03ab76eaab00f059dd4032bb2f537f
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6229 Differential Revision: D19191195 Pulled By: ltamasi fbshipit-source-id: 2a3c4785299641a46e022fc012460b759a689fce
-
由 Levi Tamasi 提交于
Summary: The patch adds support for BlobDB to `db_stress`. Note that BlobDB currently does not support (amongst other features) Column Families or the `SingleDelete` API, so for now, those should be disabled on the command line when running `db_stress` in BlobDB mode (using `-column_families=1` and `-nooverwritepercent=0`, respectively). Also, some BlobDB features that do not go well with the verification logic in `db_stress` like TTL and FIFO eviction are not supported currently. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6230 Test Plan: ``` ./db_stress -max_key=100000 -use_blob_db -column_families=1 -nooverwritepercent=0 -reopen=1 -blob_db_file_size=1000000 -target_file_size_base=1000000 -blob_db_enable_gc -blob_db_gc_cutoff=0.1 -blob_db_min_blob_size=10 -blob_db_bytes_per_sync=16384 ``` Differential Revision: D19191476 Pulled By: ltamasi fbshipit-source-id: 35840452af8c5e6095249c7fd9a53a119a0985fc
-
由 Yanqin Jin 提交于
Summary: Currently, db_stress performs verification by calling `VerifyDb()` at the end of test and optionally before tests start. In case of corruption or incorrect result, it will be too late. This PR adds more verification in two ways. 1. For cf consistency test, each test thread takes a snapshot and verifies every N ops. N is configurable via `-verify_db_one_in`. This option is not supported in other stress tests. 2. For cf consistency test, we use another background thread in which a secondary instance periodically tails the primary (interval is configurable). We verify the secondary. Once an error is detected, we terminate the test and report. This does not affect other stress tests. Test plan (devserver) ``` $./db_stress -test_cf_consistency -verify_db_one_in=0 -ops_per_thread=100000 -continuous_verification_interval=100 $./db_stress -test_cf_consistency -verify_db_one_in=1000 -ops_per_thread=10000 -continuous_verification_interval=0 $make crash_test ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6173 Differential Revision: D19047367 Pulled By: riversand963 fbshipit-source-id: aeed584ad71f9310c111445f34975e5ab47a0615
-
- 20 12月, 2019 7 次提交
-
-
由 Levi Tamasi 提交于
Summary: BlobDB currently only supports using the default column family. The earlier code enforces this by comparing the `ColumnFamilyHandle` passed to the `Get`/`Put`/etc. call with the handle returned by `DefaultColumnFamily` (which, at the end of the day, comes from `DBImpl::default_cf_handle_`). Since other `ColumnFamilyHandle`s can also point to the default column family, this can reject legitimate requests as well. (As an example, with the earlier code, the handle returned by `BlobDB::Open` cannot actually be used in API calls.) The patch fixes this by comparing only the IDs of the column family handles instead of the pointers themselves. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6226 Test Plan: `make check` Differential Revision: D19187461 Pulled By: ltamasi fbshipit-source-id: 54ce2e12ebb1f07e6d1e70e3b1e0213dfa94bda2
-
由 Peter Dillinger 提交于
Summary: The new Python syntax check could fail if external entities were cloned or symlinked to a subdir in a rocksdb git clone. (E.g. Facebook internal LITE build.) Only look for Python files in specific subdirs Pull Request resolved: https://github.com/facebook/rocksdb/pull/6225 Test Plan: python tools/check_all_python.py (still 34 files checked) Reviewed By: gfosco Differential Revision: D19186110 Pulled By: pdillinger fbshipit-source-id: 1fefa54e36b32cd5d96d3d1a43e8a2a694c22ea5
-
由 sdong 提交于
Summary: Right now BlockBasedTable::ApproximateSize() uses default setting about whether to use total order seek. There is no reason for that. There is no reason to do any filtering for approximate size boundary key, and it may introduce bugs. Disable it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6222 Test Plan: Run existing tests Differential Revision: D19184787 fbshipit-source-id: 64180660bd2800914fff75104172b61c06f0b1c9
-
由 Peter Dillinger 提交于
Summary: Complete some refactoring called for in https://github.com/facebook/rocksdb/issues/6148. Somehow I got some 'make format' in here for files I didn't change, but that should be OK. (I'm not sure why "hide whitespace changes" doesn't seem to help in review.) Not addressed in this PR: some operations simply print to stdout rather than failing on discovering a bad status or inconsistency. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6195 Differential Revision: D19131067 Pulled By: pdillinger fbshipit-source-id: 4f416e6b792023989ef119f385fe122426cb825d
-
由 Maysam Yabandeh 提交于
Summary: This reverts commit 54f9092b. It making our daily stress tests fail. Revert it until the issues are fixed. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6220 Differential Revision: D19179881 Pulled By: maysamyabandeh fbshipit-source-id: 99de0eaf776567fa81110b9ad2608234a16083ce
-
由 Peter Dillinger 提交于
Summary: We're seeing assertion violations like this in crash test: db_stress: table/block_based/block_based_table_reader.cc:4129: virtual uint64_t rocksdb::BlockBasedTable::ApproximateSize(const rocksdb::Slice&, const rocksdb::Slice&, rocksdb::TableReaderCaller): Assertion `end_offset >= start_offset' failed.*** And ApproximateSize appears only to be called with the level_compaction_dynamic_level_bytes option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6217 Test Plan: temporarily put an assert(false) in ApproximateSize and briefly run 'make crash_test' Differential Revision: D19179174 Pulled By: pdillinger fbshipit-source-id: 506e6549aea0da19b363a1a6da04373c364d92e4
-
由 Peter Dillinger 提交于
Summary: Adds a python script to syntax check all python files in the repository and report any errors. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6209 Test Plan: 'make check' with and without seeded syntax errors. Also look for "No syntax errors in 34 .py files" on success, and in java_test CI output Differential Revision: D19166756 Pulled By: pdillinger fbshipit-source-id: 537df464b767260d66810b4cf4c9808a026c58a4
-
- 19 12月, 2019 7 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Beside extending index_type to kHashSearch, it clarifies in the code base that this feature is incompatible with index_block_restart_interval > 1. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6210 Test Plan: ``` make -j32 crash_test Differential Revision: D19166567 Pulled By: maysamyabandeh fbshipit-source-id: 3aaf75a70a8b462d372d43aac69dbd10df303ec7
-
由 Levi Tamasi 提交于
Summary: The patch makes it possible to set the BlobDB configuration option `garbage_collection_cutoff` on the command line. In addition, it changes the `db_bench` code so that the default values of BlobDB related parameters are taken from the defaults of the actual BlobDB configuration options (note: this changes the the default of `blob_db_bytes_per_sync`). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6211 Test Plan: Ran `db_bench` with various values of the new parameter. Differential Revision: D19166895 Pulled By: ltamasi fbshipit-source-id: 305ccdf0123b9db032b744715810babdc3e3b7d5
-
由 sdong 提交于
Summary: level passed into ColumnFamilyData::CalculateSSTWriteHint() can be smaller than base_level in current version, which would cause overflow. We see ubsan complains: db/compaction/compaction_job.cc:1511:39: runtime error: load of value 4294967295, which is not a valid value for type 'Env::WriteLifeTimeHint' and I hope this commit fixes it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6212 Test Plan: Run existing tests and see them to pass. Differential Revision: D19168442 fbshipit-source-id: bf8fd86f85478ecfa7556db46dc3242de8c83dc9
-
由 Peter Dillinger 提交于
Summary: Should fix Travis build error that randomly showed up upon using Java 13 version of javadoc. AdvancedColumnFamilyOptionsInterface.java:257: error: unexpected heading used: <H2>, compared to implicit preceding heading: <H3> According to this reference https://bugs.openjdk.java.net/browse/JDK-8220379 it should work to start at h4, but that didn't work, so avoiding headings should be fine. Also fix Java EnvironmentTest for JDK13. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6208 Test Plan: Travis run on PR (don't have Java 13 handy) Differential Revision: D19163105 Pulled By: pdillinger fbshipit-source-id: 4a9419cbe7ef780fba771b8a1508e1ea80d17b3e
-
由 Jermy Li 提交于
Summary: improve https://github.com/facebook/rocksdb/issues/6147 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6180 Differential Revision: D19148936 fbshipit-source-id: f691c9879fd51d54e96c1a99670cf85ca4485a89
-
由 sdong 提交于
Summary: Right now, sometimes file prefetching is still on when mmap is enabled. This causes bug of reading wrong data. In this commit, we remove all those possible paths. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6206 Test Plan: make crash_test with compaction_readahead_size, which used to fail. RUn all existing tests. Differential Revision: D19149429 fbshipit-source-id: 9e18ea8c566e416aac9647bdd05afe596634791b
-
由 Peter Dillinger 提交于
Summary: Fixes syntax error from https://github.com/facebook/rocksdb/pull/6203 Pull Request resolved: https://github.com/facebook/rocksdb/pull/6207 Test Plan: make blackbox_crash_test -> no more syntax error Differential Revision: D19161752 Pulled By: pdillinger fbshipit-source-id: b3032f296041ab56307762622b9ef6c03a8379aa
-
- 18 12月, 2019 5 次提交
-
-
由 Zhichao Cao 提交于
Summary: buf_offset does not need to get the value from req.len for othe final block. It can cause test fail for clan_analyze. Remove it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6204 Test Plan: pass make asan_check Differential Revision: D19145335 Pulled By: zhichao-cao fbshipit-source-id: 8f6e74565746381b5c5ef598b97d746517b36e5b
-
由 anand76 提交于
Summary: Add an option to db_stress, verify_checksum_one_in, to call DB::VerifyChecksum() once every N ops. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6203 Differential Revision: D19145753 Pulled By: anand1976 fbshipit-source-id: d09edf21f309ad53aa40dd25b7a563d50665fd8b
-
由 Mike Kolupaev 提交于
Summary: The bad code was: ``` mutex.Lock(); // `mutex` protects `container` for (auto& x : container) { mutex.Unlock(); // do stuff to x mutex.Lock(); } ``` It's incorrect because both `x` and the iterator may become invalid if another thread modifies the container while this thread is not holding the mutex. Broken by https://github.com/facebook/rocksdb/pull/5796 - it replaced a `while (!container.empty())` loop with a `for (auto x : container)`. (RocksDB code does a lot of such unlocking+re-locking of mutexes, and this type of bugs comes up a lot :/ ) Pull Request resolved: https://github.com/facebook/rocksdb/pull/6193 Test Plan: Ran some logdevice integration tests that were crashing without this fix. Differential Revision: D19116874 Pulled By: al13n321 fbshipit-source-id: 9672bc4227c1b68f46f7436db2b96811adb8c703
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/6202 Differential Revision: D19144158 Pulled By: ltamasi fbshipit-source-id: 3e2522ced458568e3a2a045663704e30ab0ac223
-
由 sdong 提交于
Summary: Fix two crash test issues: 1. sync mode should not run with disable_wal=true 2. disable "compaction_readahead_size" for now. With it on, some block checksum verification failure will happen in compaction paths. Not sure why, but disable it for now to keep the test clean. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6200 Test Plan: Run "make crash_test" and "make crash_test_with_atomic_flush" and see it runs way longer than before the fix without failing. Differential Revision: D19143493 fbshipit-source-id: 438fad52fbda60aafd142e1b65578addbe7d72b1
-