- 16 11月, 2017 3 次提交
-
-
由 Yi Wu 提交于
Summary: Add ROCKSDB_VALGRIND_RUN macro and suppress false-positive "unimplemented functionality" throw by valgrind for steam hints. Another approach would be add a valgrind suppress file. Valgrind is suppose to print the suppression when given "--gen-suppressions=all" param, which is suppose to be the content for the suppression file. But it doesn't print. Closes https://github.com/facebook/rocksdb/pull/3174 Differential Revision: D6338786 Pulled By: yiwu-arbug fbshipit-source-id: 3559efa5f3b92d40d09ad6ac82bc7b59f86c75aa
-
由 Maysam Yabandeh 提交于
Summary: Refactor the logic around WriteCallback in the write path to clarify when and how exactly we advance the sequence number and making sure it is consistent across the code. Closes https://github.com/facebook/rocksdb/pull/3168 Differential Revision: D6324312 Pulled By: maysamyabandeh fbshipit-source-id: 9a34f479561fdb2a5d01ef6d37a28908d03bbe33
-
由 Maysam Yabandeh 提交于
Summary: The sequence number was not properly advanced after a rollback marker. The patch extends the existing unit tests to detect the bug and also fixes it. Closes https://github.com/facebook/rocksdb/pull/3157 Differential Revision: D6304291 Pulled By: maysamyabandeh fbshipit-source-id: 1b519c44a5371b802da49c9e32bd00087a8da401
-
- 15 11月, 2017 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: When testing rebuilding_trx_ in MemTableInserter might still be set before the tests finishes which would cause ASAN alarms for leaks. This patch deletes the pointers in MemTableInserter destructor. Closes https://github.com/facebook/rocksdb/pull/3162 Differential Revision: D6317113 Pulled By: maysamyabandeh fbshipit-source-id: a68be70709a4fff7ac2b768660119311968f9c21
-
- 14 11月, 2017 2 次提交
-
-
由 Yi Wu 提交于
Summary: We hit "Illegal instruction" error in regression test with "shlx" instruction. Setting PORTABLE=1 to resolve it. Closes https://github.com/facebook/rocksdb/pull/3165 Differential Revision: D6321972 Pulled By: yiwu-arbug fbshipit-source-id: cc9fe0dbd4698d1b66a750a0b062f66899862719
-
由 Yi Wu 提交于
Summary: The current implementation of PinnableSlice move assignment have an issue #3163. We are moving away from it instead of try to get the move assignment right, since it is too tricky. Closes https://github.com/facebook/rocksdb/pull/3164 Differential Revision: D6319201 Pulled By: yiwu-arbug fbshipit-source-id: 8f3279021f3710da4a4caa14fd238ed2df902c48
-
- 12 11月, 2017 2 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: This patch clarifies and refactors the logic around tracked keys in transactions. Closes https://github.com/facebook/rocksdb/pull/3140 Differential Revision: D6290258 Pulled By: maysamyabandeh fbshipit-source-id: 03b50646264cbcc550813c060b180fc7451a55c1
-
由 Maysam Yabandeh 提交于
Summary: Add tests to ensure that WritePrepared and WriteCommitted policies are cross compatible when the db WAL is empty. This is important when the admin want to switch between the policies. In such case, before the switch the admin needs to empty the WAL by i) committing/rollbacking all the pending transactions, ii) FlushMemTables Closes https://github.com/facebook/rocksdb/pull/3118 Differential Revision: D6227247 Pulled By: maysamyabandeh fbshipit-source-id: bcde3d92c1e89cda3b9cfa69f6a20af5d8993db7
-
- 11 11月, 2017 4 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Summary of changes: - Move seq_per_batch out of Options - Rename concurrent_prepare to two_write_queues - Add allocate_seq_only_for_data_ Closes https://github.com/facebook/rocksdb/pull/3136 Differential Revision: D6304458 Pulled By: maysamyabandeh fbshipit-source-id: 08e685bfa82bbc41b5b1c5eb7040a8ca6e05e58c
-
由 Andrew Kryczka 提交于
Summary: If possible, use -march or -mcpu to get enable all features available on the local CPU or architecture. Only if this is impossible, we will manually set -msse4.2. It should be safe as there'll be a warning printed if `USE_SSE` is set and the provided flags are insufficient to support SSE4.2. Closes https://github.com/facebook/rocksdb/pull/3156 Differential Revision: D6304703 Pulled By: ajkr fbshipit-source-id: 030a53491263300cae7fafb429114d87acc828ef
-
由 Dmitri Smirnov 提交于
Summary: Add per-exe execution capability Add fix parsing of groups/tests Add timer test exclusion Fix unit tests Ifdef threadpool specific tests that do not pass on Vista threadpool. Remove spurious outout from prefix_test so test case listing works properly. Fix not using standard test directories results in file creation errors in sst_dump_test. BlobDb fixes: In C++ end() iterators can not be dereferenced. They are not valid. When deleting blob_db_ set it to nullptr before any other code executes. Not fixed:. On Windows you can not delete a file while it is open. [ RUN ] BlobDBTest.ReadWhileGC d:\dev\rocksdb\rocksdb\utilities\blob_db\blob_db_test.cc(75): error: DestroyBlobDB(dbname_, options, bdb_options) IO error: Failed to delete: d:/mnt/db\testrocksdb-17444/blob_db_test/blob_dir/000001.blob: Permission denied d:\dev\rocksdb\rocksdb\utilities\blob_db\blob_db_test.cc(75): error: DestroyBlobDB(dbname_, options, bdb_options) IO error: Failed to delete: d:/mnt/db\testrocksdb-17444/blob_db_test/blob_dir/000001.blob: Permission denied write_batch Should not call front() if there is a chance the container is empty Closes https://github.com/facebook/rocksdb/pull/3152 Differential Revision: D6293274 Pulled By: sagar0 fbshipit-source-id: 318c3717c22087fae13b18715dffb24565dbd956
-
由 Shaohua Li 提交于
Summary: Add a simple policy for NVMe write time life hint Closes https://github.com/facebook/rocksdb/pull/3095 Differential Revision: D6298030 Pulled By: shligit fbshipit-source-id: 9a72a42e32e92193af11599eb71f0cf77448e24d
-
- 10 11月, 2017 2 次提交
-
-
由 kapitan-k 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3151 Differential Revision: D6293861 Pulled By: ajkr fbshipit-source-id: f8db0a71509d1cd8237f2d377bf9e1bb0464bdbf
-
由 Andrew Kryczka 提交于
Summary: The previous compression type selection caused unexpected behavior when the base level was also the bottommost level. The following sequence of events could happen: - full compaction generates files with `bottommost_compression` type - now base level is bottommost level since all files are in the same level - any compaction causes files to be rewritten `compression_per_level` type since bottommost compression didn't apply to base level I changed the code to make bottommost compression apply to base level. Closes https://github.com/facebook/rocksdb/pull/3141 Differential Revision: D6264614 Pulled By: ajkr fbshipit-source-id: d7aaa8675126896684154a1f2c9034d6214fde82
-
- 09 11月, 2017 2 次提交
-
-
由 Yi Wu 提交于
Summary: A race condition will happen when: * a user thread writes a value, but it hits the write stop condition because there are too many un-flushed memtables, while holding blob_db_impl.write_mutex_. * Flush is triggered and call flush begin listener and try to acquire blob_db_impl.write_mutex_. Fixing it. Closes https://github.com/facebook/rocksdb/pull/3149 Differential Revision: D6279805 Pulled By: yiwu-arbug fbshipit-source-id: 0e3c58afb78795ebe3360a2c69e05651e3908c40
-
由 Yi Wu 提交于
Summary: `compression` shadow the method name in `BlobFile`. Rename it. Closes https://github.com/facebook/rocksdb/pull/3148 Differential Revision: D6274498 Pulled By: yiwu-arbug fbshipit-source-id: 7d293596530998b23b6b8a8940f983f9b6343a98
-
- 08 11月, 2017 3 次提交
-
-
由 Yi Wu 提交于
Summary: To fix the issue of failing to decompress existing value after reopen DB with a different compression settings. Closes https://github.com/facebook/rocksdb/pull/3142 Differential Revision: D6267260 Pulled By: yiwu-arbug fbshipit-source-id: c7cf7f3e33b0cd25520abf4771cdf9180cc02a5f
-
由 Andrew Kryczka 提交于
Summary: - moved existing compression options to `InitializeOptionsGeneral` since they cannot be set through options file - added flag for `zstd_max_train_bytes` which was recently introduced by #3057 Closes https://github.com/facebook/rocksdb/pull/3128 Differential Revision: D6240460 Pulled By: ajkr fbshipit-source-id: 27dbebd86a55de237ba6a45cc79cff9214e82ebc
-
由 Andrew Kryczka 提交于
Summary: Without this option, running the compact benchmark on a DB containing only bottommost files simply returned immediately. Closes https://github.com/facebook/rocksdb/pull/3138 Differential Revision: D6256660 Pulled By: ajkr fbshipit-source-id: e3b64543acd503d821066f4200daa201d4fb3a9d
-
- 07 11月, 2017 1 次提交
-
-
由 Manuel Ung 提交于
Summary: Adds two new counters: `key_lock_wait_count` counts how many times a lock was blocked by another transaction and had to wait, instead of being granted the lock immediately. `key_lock_wait_time` counts the time spent acquiring locks. Closes https://github.com/facebook/rocksdb/pull/3107 Differential Revision: D6217332 Pulled By: lth fbshipit-source-id: 55d4f46da5550c333e523263422fd61d6a46deb9
-
- 04 11月, 2017 6 次提交
-
-
由 Yi Wu 提交于
Summary: After move assignment, we need to re-initialized the moved PinnableSlice. Also update blob_db_impl.cc to not reuse the moved PinnableSlice since it is supposed to be in an undefined state after move. Closes https://github.com/facebook/rocksdb/pull/3127 Differential Revision: D6238585 Pulled By: yiwu-arbug fbshipit-source-id: bd99f2e37406c4f7de160c7dee6a2e8126bc224e
-
由 Sagar Vemuri 提交于
Summary: While investigating the usage of `new_table_iterator_nanos` perf counter, I saw some code was wrapper around with unnecessary status check ... so removed it. Closes https://github.com/facebook/rocksdb/pull/3120 Differential Revision: D6229181 Pulled By: sagar0 fbshipit-source-id: f8a44fe67f5a05df94553fdb233b21e54e88cc34
-
由 Prashant D 提交于
Summary: util/concurrent_arena.h: CID 1396145 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 2. uninit_member: Non-static class member free_begin_ is not initialized in this constructor nor in any functions that it calls. 94 Shard() : allocated_and_unused_(0) {} util/dynamic_bloom.cc: 1. Condition hash_func == NULL, taking true branch. CID 1322821 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 3. uninit_member: Non-static class member data_ is not initialized in this constructor nor in any functions that it calls. 47 hash_func_(hash_func == nullptr ? &BloomHash : hash_func) {} 48 util/file_reader_writer.h: 204 private: 205 AlignedBuffer buffer_; member_not_init_in_gen_ctor: The compiler-generated constructor for this class does not initialize buffer_offset_. 206 uint64_t buffer_offset_; CID 1418246 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) member_not_init_in_gen_ctor: The compiler-generated constructor for this class does not initialize buffer_len_. 207 size_t buffer_len_; 208}; util/thread_local.cc: 341#endif CID 1322795 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 3. uninit_member: Non-static class member pthread_key_ is not initialized in this constructor nor in any functions that it calls. 342} 40struct ThreadData { 2. uninit_member: Non-static class member next is not initialized in this constructor nor in any functions that it calls. CID 1400668 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR) 4. uninit_member: Non-static class member prev is not initialized in this constructor nor in any functions that it calls. 41 explicit ThreadData(ThreadLocalPtr::StaticMeta* _inst) : entries(), inst(_inst) {} 42 std::vector<Entry> entries; 1. member_decl: Class member declaration for next. 43 ThreadData* next; 3. member_decl: Class member declaration for prev. 44 ThreadData* prev; 45 ThreadLocalPtr::StaticMeta* inst; 46}; Closes https://github.com/facebook/rocksdb/pull/3123 Differential Revision: D6233566 Pulled By: sagar0 fbshipit-source-id: aa2068790ea69787a0035c0db39d59b0c25108db
-
由 Andrew Kryczka 提交于
Summary: copied from internal diff D6156261 Closes https://github.com/facebook/rocksdb/pull/3124 Differential Revision: D6230167 Pulled By: ajkr fbshipit-source-id: 17926bb1152d607556364e3aacfec0ef3c115748
-
由 Yi Wu 提交于
Summary: Fix cast from size_t to unsigned int. Closes https://github.com/facebook/rocksdb/pull/3125 Differential Revision: D6232863 Pulled By: yiwu-arbug fbshipit-source-id: 4c6131168b1faec26f7820b2cf4a09c242d323b7
-
由 Yi Wu 提交于
Summary: Fix unreleased snapshot at the end of the test. Closes https://github.com/facebook/rocksdb/pull/3126 Differential Revision: D6232867 Pulled By: yiwu-arbug fbshipit-source-id: 651ca3144fc573ea2ab0ab20f0a752fb4a101d26
-
- 03 11月, 2017 13 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Instead of using samples directly, we now support passing the samples through zstd's dictionary generator when `CompressionOptions::zstd_max_train_bytes` is set to nonzero. If set to zero, we will use the samples directly as the dictionary -- same as before. Note this is the first step of #2987, extracted into a separate PR per reviewer request. Closes https://github.com/facebook/rocksdb/pull/3057 Differential Revision: D6116891 Pulled By: ajkr fbshipit-source-id: 70ab13cc4c734fa02e554180eed0618b75255497
-
由 Andrew Kryczka 提交于
Summary: Previously setting `write_buffer_size` with `SetOptions` would only apply to new memtables. An internal user wanted it to take effect immediately, instead of at an arbitrary future point, to prevent OOM. This PR makes the memtable's size mutable, and makes `SetOptions()` mutate it. There is one case when we preserve the old behavior, which is when memtable prefix bloom filter is enabled and the user is increasing the memtable's capacity. That's because the prefix bloom filter's size is fixed and wouldn't work as well on a larger memtable. Closes https://github.com/facebook/rocksdb/pull/3119 Differential Revision: D6228304 Pulled By: ajkr fbshipit-source-id: e44bd9d10a5f8c9d8c464bf7436070bb3eafdfc9
-
由 Zhongyi Xie 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/3121 Differential Revision: D6229415 Pulled By: miasantreble fbshipit-source-id: 57c7ad2fddf5dd6b8d7e3aaf6f62348151327dfb
-
由 Prashant D 提交于
Summary: include/rocksdb/metadata.h: struct ColumnFamilyMetaData { CID 1322804 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 2. uninit_member: Non-static class member file_count is not initialized in this constructor nor in any functions that it calls. struct SstFileMetaData { 2. uninit_member: Non-static class member size is not initialized in this constructor nor in any functions that it calls. 4. uninit_member: Non-static class member smallest_seqno is not initialized in this constructor nor in any functions that it calls. 6. uninit_member: Non-static class member largest_seqno is not initialized in this constructor nor in any functions that it calls. 8. uninit_member: Non-static class member num_reads_sampled is not initialized in this constructor nor in any functions that it calls. CID 1322807 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 10. uninit_member: Non-static class member being_compacted is not initialized in this constructor nor in any functions that it calls. include/rocksdb/sst_file_writer.h: struct ExternalSstFileInfo { 2. uninit_member: Non-static class member sequence_number is not initialized in this constructor nor in any functions that it calls. 4. uninit_member: Non-static class member file_size is not initialized in this constructor nor in any functions that it calls. 6. uninit_member: Non-static class member num_entries is not initialized in this constructor nor in any functions that it calls. CID 1351697 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 8. uninit_member: Non-static class member version is not initialized in this constructor nor in any functions that it calls. 31 ExternalSstFileInfo() {} include/rocksdb/utilities/transaction.h: explicit Transaction(const TransactionDB* db) {} 2. uninit_member: Non-static class member log_number_ is not initialized in this constructor nor in any functions that it calls. CID 1396133 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR) 4. uninit_member: Non-static class member field txn_state_._M_i is not initialized in this constructor nor in any functions that it calls. 473 Transaction() {} Closes https://github.com/facebook/rocksdb/pull/3100 Differential Revision: D6227651 Pulled By: sagar0 fbshipit-source-id: 5caa4a2cf9471d1f9c3c073f81473636e1f0aa14
-
由 Yi Wu 提交于
Summary: After adding expiration to blob index in #3066, we are now able to add a compaction filter to cleanup expired blob index entries. Closes https://github.com/facebook/rocksdb/pull/3090 Differential Revision: D6183812 Pulled By: yiwu-arbug fbshipit-source-id: 9cb03267a9702975290e758c9c176a2c03530b83
-
由 Sagar Vemuri 提交于
Summary: This options was introduced in the C++ API in #1953 . Closes https://github.com/facebook/rocksdb/pull/3064 Differential Revision: D6139010 Pulled By: sagar0 fbshipit-source-id: 164de11d539d174cf3afe7cd40e667049f44b0bc
-
由 Yi Wu 提交于
Summary: Blob db will keep blob file if data in the file is visible to an active snapshot. Before this patch it checks whether there is an active snapshot has sequence number greater than the earliest sequence in the file. This is problematic since we take snapshot on every read, if it keep having reads, old blob files will not be cleanup. Change to check if there is an active snapshot falls in the range of [earliest_sequence, obsolete_sequence) where obsolete sequence is 1. if data is relocated to another file by garbage collection, it is the latest sequence at the time garbage collection finish 2. otherwise, it is the latest sequence of the file Closes https://github.com/facebook/rocksdb/pull/3087 Differential Revision: D6182519 Pulled By: yiwu-arbug fbshipit-source-id: cdf4c35281f782eb2a9ad6a87b6727bbdff27a45
-
由 Yi Wu 提交于
Summary: Add an option to enable/disable auto garbage collection, where we keep counting how many keys have been evicted by either deletion or compaction and decide whether to garbage collect a blob file. Default disable auto garbage collection for now since the whole logic is not fully tested and we plan to make major change to it. Closes https://github.com/facebook/rocksdb/pull/3117 Differential Revision: D6224756 Pulled By: yiwu-arbug fbshipit-source-id: cdf53bdccec96a4580a2b3a342110ad9e8864dfe
-
由 Yi Wu 提交于
Summary: The test intent to wait until key being overwritten until proceed with garbage collection. It failed to wait for `PutUntil` finally finish. Fixing it. Closes https://github.com/facebook/rocksdb/pull/3116 Differential Revision: D6222833 Pulled By: yiwu-arbug fbshipit-source-id: fa9b57a772b92a66cf250b44e7975c43f62f45c5
-
由 Sagar Vemuri 提交于
Summary: Evict oldest blob file and put it in obsolete_files list when close to blob db size limit. The file will be delete when the `DeleteObsoleteFiles` background job runs next time. For now I set `kEvictOldestFileAtSize` constant, which controls when to evict the oldest file, at 90%. It could be tweaked or made into an option if really needed; I didn't want to expose it as an option pre-maturely as there are already too many :) . Closes https://github.com/facebook/rocksdb/pull/3094 Differential Revision: D6187340 Pulled By: sagar0 fbshipit-source-id: 687f8262101b9301bf964b94025a2fe9d8573421
-
由 Prashant D 提交于
Summary: The num() might return cur_num as 0 and we are making sure that cur_num will not be 0 down the path. The mult variable is being set to 100.0/cur_num which makes program crash when cur_num is 0. Closes https://github.com/facebook/rocksdb/pull/3105 Differential Revision: D6222594 Pulled By: ajkr fbshipit-source-id: 986154709897ff4dbbeb0e8aa81eb8c0b2a2db76
-
由 Maysam Yabandeh 提交于
Summary: This patch will remove the existing comments that declare partitioning indexes and filters as experimental. Closes https://github.com/facebook/rocksdb/pull/3115 Differential Revision: D6222227 Pulled By: maysamyabandeh fbshipit-source-id: 6179ec43b22c518494051b674d91c9e1b54d4ac0
-
由 Maysam Yabandeh 提交于
Summary: Move WritePreparedTxnDB from pessimistic_transaction_db.h to its own header, write_prepared_txn_db.h Closes https://github.com/facebook/rocksdb/pull/3114 Differential Revision: D6220987 Pulled By: maysamyabandeh fbshipit-source-id: 18893fb4fdc6b809fe117dabb544080f9b4a301b
-
- 02 11月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: It's also defined in db/dbformat.cc per 7fe3b328 Closes https://github.com/facebook/rocksdb/pull/3111 Differential Revision: D6219140 Pulled By: ajkr fbshipit-source-id: 0f2b14e41457334a4665c6b7e3f42f1a060a0f35
-