- 16 1月, 2019 4 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: The AdvanceMaxEvictedSeq algorithm assumes that new snapshots always have sequence number larger than the last max_evicted_seq_. To enforce this assumption we make two changes: i) max is not advanced beyond the last published seq, with the exception that the evicted commit entry itself is not published yet, which is quite rare. ii) When obtaining the snapshot if the max_evicted_seq_ is not published yet, commit a dummy entry so that it waits for it to be published and also increased the latest published seq by one above the max. To test these non-realistic corner cases we create a commit cache with size 1 so that every single commit results into eviction. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4886 Differential Revision: D13685270 Pulled By: maysamyabandeh fbshipit-source-id: 5461bc09c2a9b75798bfcb9853a256c81cdac0b0
-
由 Siying Dong 提交于
Summary: Right now the error mesage when options.wal_dir doesn't exist is not helpful to users. Be more specific Pull Request resolved: https://github.com/facebook/rocksdb/pull/4874 Differential Revision: D13642425 Pulled By: siying fbshipit-source-id: 9a3172ed0f799af233b0f3b2e5e35bc7ce04c7b5
-
由 Sagar Vemuri 提交于
Summary: - Corrected a comment asserting that the values "smaller" than a min_blob_size will be inlined in the base db. - Also fixed the type of ttl_range_secs while dumping blobdb options. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4887 Differential Revision: D13680163 Pulled By: sagar0 fbshipit-source-id: 306c8cf2daa52210ffc334a6924ef44ffdedf887
-
由 Yi Wu 提交于
Summary: When prepared_txns_ heap is empty, SmallestUnCommittedSeq() should check delayed_prepared_ set as well. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4867 Differential Revision: D13632134 Pulled By: maysamyabandeh fbshipit-source-id: b0423bb0a58dc95f1e636d5ed3f6e619df801fb7
-
- 12 1月, 2019 5 次提交
-
-
由 Yanqin Jin 提交于
Summary: If we do not do this, then reading MutableCFOptions may have a race condition with SetOptions which modifies MutableCFOptions. Also reserve space in advance for vectors to avoid reallocation changing the address of its elements. Test plan ``` $make clean && make -j32 all check $make clean && COMPILE_WITH_TSAN=1 make -j32 all check $make clean && COMPILE_WITH_ASAN=1 make -j32 all check ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/4876 Differential Revision: D13644500 Pulled By: riversand963 fbshipit-source-id: 4b8112c5c819d5a2922bb61ad1521b3d2fb2fd47
-
由 Siying Dong 提交于
Summary: LDB is frequently used to exam data copied. wal_dir in option file is not modified and it usually points to the path it copied from. The user experience will be better if when ldb sees wal_dir pointed by the option file doesn't exist, rather than fail, just ignore it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4875 Differential Revision: D13643173 Pulled By: siying fbshipit-source-id: 2e64d4ea2ec49a6794b9a706b7fc1ba901128bb8
-
由 Maysam Yabandeh 提交于
Summary: Fixes a typo that made mutex_ to remain unlocked when GetSnapshotListFromDB called from WritePreparedTxnDB. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4872 Differential Revision: D13640381 Pulled By: maysamyabandeh fbshipit-source-id: 50f6600568f9092b4b43115f6ebd96e6c7388ad7
-
由 Zhongyi Xie 提交于
Summary: currently clang analyze fails with the following warning: > utilities/transactions/write_prepared_transaction_test.cc:1451:5: warning: Forming reference to null pointer ASSERT_GT(wp_db->max_evicted_seq_, 0); // max after recovery Pull Request resolved: https://github.com/facebook/rocksdb/pull/4871 Differential Revision: D13638053 Pulled By: miasantreble fbshipit-source-id: b192b0c13c411c58defc9e280b34cdfcab3fa8e3
-
由 Varadharajan 提交于
Summary: Build failing due to incorrect filename. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4870 Differential Revision: D13637205 Pulled By: sagar0 fbshipit-source-id: 72da45d51b49bce32f696532ba0656ee0dc2b89f
-
- 11 1月, 2019 1 次提交
-
-
由 Siying Dong 提交于
Summary: Remove some components that we never heard people using them. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4101 Differential Revision: D8825431 Pulled By: siying fbshipit-source-id: 97a12ad3cad4ab12c82741a5ba49669aaa854180
-
- 10 1月, 2019 2 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: The vector returned by SnapshotList::GetAll could have duplicate entries if two separate snapshots have the same sequence number. However, when this vector is used in compaction the duplicate entires are of no use and could be safely ignored. Moreover not having duplicate entires simplifies reasoning in the compaction_iterator.cc code. For example when searching for the previous_snap we currently use the snapshot before the current one but the way the code uses that it expects it to be also less than the current snapshot, which would be simpler to read if there is no duplicate entry in the snapshot list. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4860 Differential Revision: D13615502 Pulled By: maysamyabandeh fbshipit-source-id: d45bf01213ead5f39db811f951802da6fcc3332b
-
由 Yanqin Jin 提交于
Summary: as titled. Currently it's possible to create a local object of type PerfContext since it's part of public API. Then it's safe to initialize the two members to 0. If PerfContext is created as thread-local object, then all members are zero-initialized according to C++ standard. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4859 Differential Revision: D13614504 Pulled By: riversand963 fbshipit-source-id: 406ff548e105a074f379ad1054d56fece5f524a0
-
- 09 1月, 2019 4 次提交
-
-
由 Yanqin Jin 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/4857 Differential Revision: D13602688 Pulled By: riversand963 fbshipit-source-id: 993419a6afb982a7a701ff71daebebb4b4a6b265
-
由 Maysam Yabandeh 提交于
Summary: Previously IsInSnapshot assumed that the snapshot is valid at the time that the function is called. However there are cases where that might not be valid. Example is background compactions where the compaction algorithm operates with a list of snapshots some of which might be released by the time they are being passed to IsInSnapshot. The patch make two changes to enable the caller to tell difference: i) any live snapshot below max is added to max_committed_seq_, which allows IsInSnapshot to confidently tell whether the passed snapshot is invalid if it below max, ii) extends IsInSnapshot API with a "released" variable that is set true when IsInSnapshot find no such snapshot below max and also find no other way to give a certain return value. In such cases the return value is true but the caller should also check the "released" boolean after the call. In short here is the changes in the API: i) If the snapshot is valid, no change is required. ii) If the snapshot might be invalid, a reference to "released" boolean must be passed to IsInSnapshot. ii-a) If snapshot is above max, IsInSnapshot can figure the return valid using the commit cache. ii-b) otherwise if snapshot is in old_commit_map_, IsInSnapshot can use that to tell if value was visible to the snapshot. ii-c) otherwise it sets "released" to true and returns true as well. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4856 Differential Revision: D13599847 Pulled By: maysamyabandeh fbshipit-source-id: 1752be28667f886a1efec8cae5714b9b7a8f1e0f
-
由 Siying Dong 提交于
Summary: https://github.com/facebook/rocksdb/pull/3340 introduces preloading when max_open_files != -1. It doesn't preload index and filter in non-initial file loading case. This is a little bit too complicated to understand. We observed in one MyRocks use case where the filter is expected to be preloaded but is not. To simplify the use case, we simply always prefetch the index and filter. They anyway is expected to be loaded in the file verification phase anyway. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4852 Differential Revision: D13595402 Pulled By: siying fbshipit-source-id: d4d8624eb3e849e20aeb990df2100502d85aff31
-
由 Maysam Yabandeh 提交于
Summary: IsInSnapshotEmptyMapTest tests that IsInSnapshot returns correct value for existing data after a recovery, where max is not zero and yet commit cache is empty. The existing test was preliminary which is improved in this patch. It also increases the db sequence after recovery so that there the snapshot immediately taken after recovery would have a sequence number different than that of max_evicted_seq. This simplifies the logic in IsInSnapshot by not having to consider the special case that an old snapshot might be equal to max_evicted_seq and yet not present in old_commit_map. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4853 Differential Revision: D13595223 Pulled By: maysamyabandeh fbshipit-source-id: 77c12ca8a3f61a47479a93bef2038ff502dc3322
-
- 08 1月, 2019 4 次提交
-
-
由 Yanqin Jin 提交于
Summary: as titled. We can remove the assersion because we do not perform verification in AtomicFlushStressTest::TestCheckpoint for similar reasons to TestGet, TestPut, etc. Therefore, we override TestCheckpoint in AtomicFlushStressTest so that the assertion `rand_column_families.size() == rand_keys.size()' is removed, and we do not verify the DB in this function. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4846 Differential Revision: D13583377 Pulled By: riversand963 fbshipit-source-id: 03647f3da67e27a397413fd666e3bb43003bf596
-
由 Maysam Yabandeh 提交于
Summary: The rollback algorithm in WritePrepared transactions requires reading the values before the transaction start. Currently it uses the prepare_seq -1 as the snapshot sequence number for the read. This is not correct since the passed sequence number must be for a valid snapshot. The patch fixes it by passing kMaxSequenceNumber instead. This is fine since all the writes done by the aborted transaction will be skipped during the read anyway. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4851 Differential Revision: D13592773 Pulled By: maysamyabandeh fbshipit-source-id: ff1bf92ea9909d4cccb173bdff49febc0e9eb7a2
-
由 tom wang 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/4850 Differential Revision: D13591940 Pulled By: sagar0 fbshipit-source-id: 617794e0a41d0f4554d40871180b061e84189fc5
-
由 Philip Jameson 提交于
Reviewed By: ttsugriy Differential Revision: D13583867 fbshipit-source-id: 8f218a9ffd9807d386ba0adc966af2a9a48ac64c
-
- 05 1月, 2019 2 次提交
-
-
由 Yi Wu 提交于
Summary: In compaction iterator, if the next value of single delete is a blob value, it should not treated as mismatch. This is only a minor fix and doesn't affect correctness. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4848 Differential Revision: D13585812 Pulled By: yiwu-arbug fbshipit-source-id: 0ff6223fa03a644ac9fd8a2d77f9d6711d0a62b0
-
由 Andrew Kryczka 提交于
Summary: Previously for point lookup we decided which file to look into based on user key overlap only. We also did not truncate range tombstones in the point lookup code path. These two ideas did not interact well in cases like this: - L1 has range tombstone [a, c)#1 and point key b#2. The data is split between file1 with range [a#1,1, b#72057594037927935,15], and file2 with range [b#2, c#1]. - L1's file2 gets compacted to L2. - User issues `Get()` for b#3. - L1's file1 is opened and the range tombstone [a, c)#1 is found for b, while no point-key for b is found in L1. - `Get()` assumes that the range tombstone must cover all data in that range in lower levels, so short circuits and returns `NotFound`. The solution to this problem is to not look into files that only overlap with the point lookup at a range tombstone sentinel endpoint. In the above example, this would mean not opening L1's file1 or its tombstones during the `Get()`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4829 Differential Revision: D13561355 Pulled By: ajkr fbshipit-source-id: a13c21c816870a2f5d32a48af6dbd719a7d9d19f
-
- 04 1月, 2019 6 次提交
-
-
由 Yanqin Jin 提交于
Summary: as titled. Since different bg flush threads can flush different sets of column families (due to column family creation and drop), we decide not to let one thread perform atomic flush result installation for other threads. Bg flush threads will install their atomic flush results sequentially to MANIFEST, using a conditional variable, i.e. atomic_flush_install_cv_ to coordinate. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4791 Differential Revision: D13498930 Pulled By: riversand963 fbshipit-source-id: dd7482fc41f4bd22dad1e1ef7d4764ef424688d7
-
由 Yi Wu 提交于
Summary: Declare Jemalloc non-standard APIs as weak symbols, so that if Jemalloc is linked with the binary, these symbols will be replaced by Jemalloc's, otherwise they will be nullptr. This is similar to how folly detect jemalloc, but we assume the main program use jemalloc as long as jemalloc is linked: https://github.com/facebook/folly/blob/master/folly/memory/Malloc.h#L147 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4844 Differential Revision: D13574934 Pulled By: yiwu-arbug fbshipit-source-id: 7ea871beb1be7d5a1259cc38f9b78078793db2db
-
由 DorianZheng 提交于
Summary: The original implementation has two problems: 1. https://github.com/facebook/rocksdb/blob/f0dda35d7de1fd56e0b7c96376ca8aff2a6364fd/db/db_impl_write.cc#L478 https://github.com/facebook/rocksdb/blob/f0dda35d7de1fd56e0b7c96376ca8aff2a6364fd/db/write_thread.h#L231 If the callback status of leader of the write_group fails, then the whole write_group will not write to WAL, this may cause data loss. 2. https://github.com/facebook/rocksdb/blob/f0dda35d7de1fd56e0b7c96376ca8aff2a6364fd/db/write_thread.h#L130 The annotation says that Writer.status is the status of memtable inserter, but the original implementation use it for another case which is not consistent with the original design. Looks like we can still reuse Writer.status, but we should modify the annotation, so Writer.status is not only the status of memtable inserter. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4838 Differential Revision: D13574070 Pulled By: yiwu-arbug fbshipit-source-id: a2a2aefcfd329c4c6a91652bf090aaf1ce119c4b
-
由 Huachao Huang 提交于
Summary: The current implementation hardcode the default options in different places, which makes it impossible to support other environments (like encrypted environment). Pull Request resolved: https://github.com/facebook/rocksdb/pull/4839 Differential Revision: D13573578 Pulled By: sagar0 fbshipit-source-id: 76b58b4b758902798d10ff2f52d9f39abff015e7
-
由 Siying Dong 提交于
Summary: DBSSTTest.RateLimitedDelete is flakey. The root cause is not completely identified, but the compaction waiting in the test doesn't strictly wait for compaction cleaning to finish, which may cause test flakiness. Fix it first and see whether the failures still happen. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4840 Differential Revision: D13567273 Pulled By: siying fbshipit-source-id: 6fce38b912aff92a925231e7aa9bb0fef892761a
-
由 Adam Retter 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/4843 Differential Revision: D13573607 Pulled By: sagar0 fbshipit-source-id: 0e752992d1d0187cd423f47b53f9b1f80555f8cd
-
- 03 1月, 2019 7 次提交
-
-
由 Yanqin Jin 提交于
Summary: Updated stress test will support testing of db in read-only mode. The user has to make sure that only read/scan operations are enabled. This PR relies on #4681. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4690 Differential Revision: D13102741 Pulled By: riversand963 fbshipit-source-id: f5a36b34db187fe12dd355f7eda161f99d6c75e4
-
由 Andrew Kryczka 提交于
Summary: - To be consistent with the accounting of other optypes in `TableProperties`, we should count range tombstones in `TableProperties::num_entries` and `TableProperties::num_deletions`. - Updated assertions in stress test's `OnTableFileCreated` handler to accept files with range tombstones only. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4841 Differential Revision: D13568424 Pulled By: ajkr fbshipit-source-id: 0139d7806494eda20ece67ec460d2458dbbf6026
-
由 Tongliang Liao 提交于
Summary: CMake 3 already has FindZLIB. [https://cmake.org/cmake/help/v3.13/module/FindZLIB.html](https://cmake.org/cmake/help/v3.13/module/FindZLIB.html) Pull Request resolved: https://github.com/facebook/rocksdb/pull/4823 Differential Revision: D13567653 Pulled By: ajkr fbshipit-source-id: e424aac1e5d9af4ee0d293896faedf7c712f7734
-
由 Anand Ananthabhotla 提交于
Summary: Avoid locking the DB mutex in order to reference SuperVersions. Instead, we get the thread local cached SuperVersion for each column family in the list. It depends on finding a sequence number that overlaps with all the open memtables. We start with the latest published sequence number, and if any of the memtables is sealed before we can get all the SuperVersions, the process is repeated. After a few times, give up and lock the DB mutex. Tests: 1. Unit tests 2. make check 3. db_bench - TEST_TMPDIR=/dev/shm ./db_bench -use_existing_db=true -benchmarks=readrandom -write_buffer_size=4194304 -target_file_size_base=4194304 -max_bytes_for_level_base=16777216 -num=5000000 -reads=1000000 -threads=32 -compression_type=none -cache_size=1048576000 -batch_size=1 -bloom_bits=1 readrandom : 0.167 micros/op 5983920 ops/sec; 426.2 MB/s (1000000 of 1000000 found) Multireadrandom with batch size 1: multireadrandom : 0.176 micros/op 5684033 ops/sec; (1000000 of 1000000 found) Pull Request resolved: https://github.com/facebook/rocksdb/pull/4754 Differential Revision: D13363550 Pulled By: anand1976 fbshipit-source-id: 6243e8de7dbd9c8bb490a8eca385da0c855b1dd4
-
由 Faustin Lammler 提交于
Summary: Hi, Lintian, the Debian package checker complains about spelling error (spelling-error-in-binary). See https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/98380 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4827 Differential Revision: D13566362 Pulled By: riversand963 fbshipit-source-id: cd4e9212133c73b0591030de6cdedaa47575968d
-
由 Tongliang Liao 提交于
Summary: `rocksdb-shared.lib` is missing while `rocksdb.lib` and `rocksdb-shared.dll` are installed correctly. Add `ARCHIVE DESTINATION` to fix this issue. Refer to CMake doc for more details: [ https://cmake.org/cmake/help/v3.13/command/install.html#installing-targets](https://cmake.org/cmake/help/v3.13/command/install.html#installing-targets) > ARCHIVE > Static libraries are treated as ARCHIVE targets, except those marked with the FRAMEWORK property on macOS (see FRAMEWORK below.) For DLL platforms (all Windows-based systems including Cygwin), the DLL import library is treated as an ARCHIVE target. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4832 Differential Revision: D13566301 Pulled By: siying fbshipit-source-id: 56e4ef82f7d5c63bd181ddf23b691336ad77881a
-
由 Yanqin Jin 提交于
Summary: The `flush_reason` parameter in `DBImpl::InstallSuperVersionAndScheduleWork` is not used. Remove it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4816 Differential Revision: D13543218 Pulled By: riversand963 fbshipit-source-id: 8fc75d49462ce092e85aef0fe0c50936140db153
-
- 29 12月, 2018 1 次提交
-
-
由 Siying Dong 提交于
Summary: Choose to preload some files if options.max_open_files != -1. This can slightly narrow the gap of performance between options.max_open_files is -1 and a large number. To avoid a significant regression to DB reopen speed if options.max_open_files != -1. Limit the files to preload in DB open time to 16. Pull Request resolved: https://github.com/facebook/rocksdb/pull/3340 Differential Revision: D6686945 Pulled By: siying fbshipit-source-id: 8ec11bbdb46e3d0cdee7b6ad5897a09c5a07869f
-
- 27 12月, 2018 2 次提交
-
-
由 Burton Li 提交于
Summary: 1. Remove unused API SubtractCompactionTask(). 2. Assert outstanding tasks drop to zero in ConcurrentTaskLimiterImpl destructor. 3. Remove GetOutstandingTask() check from manual compaction test, as TEST_WaitForCompact() doesn't synced with 'delete prepicked_compaction' in DBImpl::BGWorkCompaction(), which may make the test flaky. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4795 Differential Revision: D13542183 Pulled By: siying fbshipit-source-id: 5eb2a47e62efe4126937149aa0df6e243ebefc33
-
由 Max 提交于
Summary: Fix some typos in comments. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4819 Differential Revision: D13548543 Pulled By: siying fbshipit-source-id: ca2e128fa47bef32892fc3627a7541fd9e2d5c3f
-
- 22 12月, 2018 2 次提交
-
-
由 faust 提交于
Summary: Lintian, the Debian package checker complains about insane-line-length-in-source-file. Line length is 278 characters (>256) Please see here the error: https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/95739 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4813 Differential Revision: D13539183 Pulled By: miasantreble fbshipit-source-id: 28ad31d1bf23a076b9e4fc9ff62fb0b4c63a65f6
-
由 Alexander Zinoviev 提交于
Summary: Add a new per level counter for block cache hits, increase it by one on every successful attempt to get an entry from cache. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4796 Differential Revision: D13513688 Pulled By: zinoale fbshipit-source-id: 104df038f1232e3356e162eb2d8ca138e34a8281
-