- 31 5月, 2019 4 次提交
-
-
由 anand76 提交于
Summary: When the --reopen option is non-zero, the DB is reopened after every ops_per_thread/(reopen+1) ops, with the check being done after every op. With MultiGet, we might do multiple ops in one iteration, which broke the logic that checked when to synchronize among the threads and reopen the DB. This PR fixes that logic. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5374 Differential Revision: D15559780 Pulled By: anand1976 fbshipit-source-id: ee6563a68045df7f367eca3cbc2500d3e26359ef
-
由 Siying Dong 提交于
Summary: There are too many types of files under util/. Some test related files don't belong to there or just are just loosely related. Mo ve them to a new directory test_util/, so that util/ is cleaner. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5377 Differential Revision: D15551366 Pulled By: siying fbshipit-source-id: 0f5c8653832354ef8caa31749c0143815d719e2c
-
由 anand76 提交于
Summary: By increasing the ratio, we ensure that all files go through background deletion and eliminate flakiness due to timing of deletions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5366 Differential Revision: D15549992 Pulled By: anand1976 fbshipit-source-id: d137375cd791fc1a802841412755d6e2b8fd7688
-
由 Zhongyi Xie 提交于
Summary: When dynamically setting options, we check the option type info and skip options that are marked deprecated. However this check is only done at top level, which results in bugs where SetOptions will corrupt option values and cause unexpected system behavior iff a deprecated second level option is set dynamically. For exmaple, the following call: ``` dbfull()->SetOptions( {{"compaction_options_fifo", "{allow_compaction=true;max_table_files_size=1024;ttl=731;}"}}); ``` was from pre 6.0 release when `ttl` was part of `compaction_options_fifo`. Now that it got moved out of `compaction_options_fifo`, this call will incorrectly set `compaction_options_fifo.max_table_files_size` to 731 (as `max_table_files_size` is the first one in `OptionsHelper::fifo_compaction_options_type_info` struct) and cause files to gett evicted much faster than expected. This PR adds verification to second level options like `compaction_options_fifo.ttl` or `compaction_options_fifo.max_table_files_size` when set dynamically, and filter out those marked as deprecated. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5367 Differential Revision: D15530998 Pulled By: miasantreble fbshipit-source-id: 818258be5c3abe09cd82d62f3c083572d70fecdd
-
- 30 5月, 2019 1 次提交
-
-
由 Siying Dong 提交于
Summary: util/ means for lower level libraries, so it's a good idea to move the files which requires knowledge to DB out. Create a file/ and move some files there. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5375 Differential Revision: D15550935 Pulled By: siying fbshipit-source-id: 61a9715dcde5386eebfb43e93f847bba1ae0d3f2
-
- 29 5月, 2019 4 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: This enables the user to set TransactionDBOptions::skip_concurrency_control so the standard `DB::Write(const WriteOptions& opts, WriteBatch* updates)` would skip the concurrency control. This would give higher throughput to the users who know their use case doesn't need concurrency control. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5330 Differential Revision: D15525932 Pulled By: maysamyabandeh fbshipit-source-id: 68421ac1ba34f549a4a8de9ce4c2dccf6fb4b06b
-
由 Maysam Yabandeh 提交于
Summary: When committing a transaction without prepare, WritePrepared simply writes the batch to db and add the commit entry to CommitCache. When two_write_queues=true, following the rule of committing only from 2nd write queue, the first write, writes the batch and the only thing the 2nd write does is to write the commit entry to CommitCache. Currently the write batch in 2nd write is set to an empty LogData entry, while the write to the WAL could simply be entirely disabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5327 Differential Revision: D15424546 Pulled By: maysamyabandeh fbshipit-source-id: 3d9ea3922d5196984c584d62a3ed57e1f7ca7b9f
-
由 Siying Dong 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5357 Differential Revision: D15522825 Pulled By: siying fbshipit-source-id: d775386b9d10c7179f5d3af2c821ed213abfacdf
-
由 Yanqin Jin 提交于
Summary: The analyzer thinks max_allowed_ space can be 0. In that case, free_space will be assigned as free_space. It fails to realize that the function call GetFreeSpace actually sets the free_space variable properly, which is possibly due to lack of inter-function call analysis. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5365 Differential Revision: D15521790 Pulled By: riversand963 fbshipit-source-id: 839d0a285a1c8773a28a385f0c3be4bb7fbe32cb
-
- 25 5月, 2019 15 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Add file and class level comments in file_reader_writer.h Pull Request resolved: https://github.com/facebook/rocksdb/pull/5355 Differential Revision: D15499020 Pulled By: sagar0 fbshipit-source-id: 925b2326885cdb4357e6a139ac65ee5e2ce1d613
-
由 Yanqin Jin 提交于
Summary: If RocksDB is configured with a positive max_allowed_space (via sst file manager), then the sst file manager should use this value instead of total free disk space to determine whether to clear the background error of space limit reached. In DBSSTTest.DBWithMaxSpaceAllowed, we configure a low space limit that is very likely lower than the free disk space of the test machine. Therefore, once the test db encounters a Status::SpaceLimit, error handler will call into sst file manager to start error recovery which may clear the bg error since disk free space is larger than reserved_disk_buffer_. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5334 Differential Revision: D15501622 Pulled By: riversand963 fbshipit-source-id: 58035efc450b062d6b28c78c322005ec3705fb47
-
由 Sagar Vemuri 提交于
Summary: Add file, class, and function level comments in trace_replay.h Pull Request resolved: https://github.com/facebook/rocksdb/pull/5359 Differential Revision: D15505318 Pulled By: sagar0 fbshipit-source-id: 181e3d4ea805fd9a33f91b89e123bbd0c1ead2ce
-
由 Zhongyi Xie 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5360 Differential Revision: D15502973 Pulled By: miasantreble fbshipit-source-id: 15b7f9d7928e771a6fac0643861173be8ba6b37a
-
由 anand76 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5343 Differential Revision: D15485831 Pulled By: anand1976 fbshipit-source-id: 8735ccfba90d7ecb3559e63f792e34527f04ed29
-
由 anand76 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5353 Differential Revision: D15497912 Pulled By: anand1976 fbshipit-source-id: 72cff2465ca342aa810f925be5a7016b938aa416
-
由 Siying Dong 提交于
Summary: Add some comments in db_impl.h. Also reordered function order a little bit so that I can add a comment to flag the area of functions implementing DB interface. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5338 Differential Revision: D15498284 Pulled By: siying fbshipit-source-id: 3d7c59c8303577fe44d13c74ae84c7ce05164f77
-
由 Vijay Nadimpalli 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5352 Differential Revision: D15498477 Pulled By: vjnadimpalli fbshipit-source-id: 08a981521848433362a56ac521c7fb83c7dd7b2a
-
由 Siying Dong 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5356 Differential Revision: D15499033 Pulled By: siying fbshipit-source-id: 069ae48669484beaf668dd90389b8743b3309dc3
-
由 Siying Dong 提交于
Summary: Document relationships of data structures declared in column_family.h Pull Request resolved: https://github.com/facebook/rocksdb/pull/5347 Differential Revision: D15496941 Pulled By: siying fbshipit-source-id: 47b37835abba26aa31a94fabea6b2775483e0ccb
-
由 Zhongyi Xie 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5346 Differential Revision: D15497679 Pulled By: miasantreble fbshipit-source-id: c10caf10293c3d9663bfb398a0d331326d1e9e67
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5354 Differential Revision: D15496645 Pulled By: ltamasi fbshipit-source-id: 1282b1ce11fbc412d3d87b2688fd0586e7bb6b85
-
由 Zhongyi Xie 提交于
Summary: att Pull Request resolved: https://github.com/facebook/rocksdb/pull/5350 Differential Revision: D15496467 Pulled By: miasantreble fbshipit-source-id: c29c0b143bf4df2040695a82be0feb9814ddb641
-
由 Zhongyi Xie 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5351 Differential Revision: D15496346 Pulled By: miasantreble fbshipit-source-id: eeb619e6bd8616003ba35b0cd4bb8050e6a8cb4d
-
由 Sagar Vemuri 提交于
Summary: Add comments to util/aligned_buffer.h Pull Request resolved: https://github.com/facebook/rocksdb/pull/5345 Differential Revision: D15496004 Pulled By: sagar0 fbshipit-source-id: 31bc6f35e88dedd74cff55febe02c9e761304f76
-
- 24 5月, 2019 8 次提交
-
-
由 haoyuhuang 提交于
Summary: RocksDB always tries to perform a hard link operation on the external SST file to ingest. This operation can fail if the external SST resides on a different device/FS, or the underlying FS does not support hard link. Currently RocksDB assumes that if the link fails, the user is willing to perform file copy, which is not true according to the post. This commit provides an option named 'failed_move_fall_back_to_copy' for users to choose which behavior they want. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5333 Differential Revision: D15457597 Pulled By: HaoyuHuang fbshipit-source-id: f3626e13f845db4f7ed970a53ec8a2b1f0d62214
-
由 Zhongyi Xie 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5344 Differential Revision: D15485431 Pulled By: miasantreble fbshipit-source-id: 9c0f6cf0d826743e743012549976705ceb8cc0c4
-
由 Zhongyi Xie 提交于
Summary: add class/function level comments to the header file Pull Request resolved: https://github.com/facebook/rocksdb/pull/5341 Differential Revision: D15485442 Pulled By: miasantreble fbshipit-source-id: 9f11e2a1cd3ce0f4990f01353d0a6f4b050615cf
-
由 Siying Dong 提交于
Summary: Simply add some comments. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5339 Differential Revision: D15485315 Pulled By: siying fbshipit-source-id: 4594b1c4c967e6bd08aa7fa08a37df3481df1938
-
由 Siying Dong 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5342 Differential Revision: D15485238 Pulled By: siying fbshipit-source-id: a56b374584cb1d815c1173907a807d90b37d4dd6
-
由 Siying Dong 提交于
Summary: Add file comment in db/db_iter.h and minor changes in other parts. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5340 Differential Revision: D15484605 Pulled By: siying fbshipit-source-id: 173771f9d5bd51303de5410ee5afd0a4af9d6572
-
由 Levi Tamasi 提交于
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5337 Differential Revision: D15482289 Pulled By: ltamasi fbshipit-source-id: 8639ca78c1b8dfcc337a742d4d81d5752f12545f
-
由 Silver Chan 提交于
Summary: when building this file using Xcode 10.2.1 in MacOSX10.14, the compiler report this error: ` rocksdb/tools/db_stress.cc:3613:33: error: implicit instantiation of undefined template 'std::__1::array<std::__1::basic_string<char>, 10>' std::array<std::string, 10> keys = {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"}; /usr/include/c++/v1/__tuple:223:64: note: template is declared here template <class _Tp, size_t _Size> struct _LIBCPP_TEMPLATE_VIS array; ^ 1 error generated. ` if including array, this error will be fixed. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5307 Differential Revision: D15475217 Pulled By: sagar0 fbshipit-source-id: b04a7658c2ca2573157028863b3a80f5ab52b9de
-
- 23 5月, 2019 2 次提交
-
-
由 Thomas Fersch 提交于
Summary: Modified FindIntraL0Compaction to stop picking more files if total amount of compensated bytes would be larger than max_compaction_bytes option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5329 Differential Revision: D15435728 Pulled By: ThomasFersch fbshipit-source-id: d118a6da88d5df8ee20944422ade37cf6b15d60c
-
由 haoyuhuang 提交于
Summary: In version_set.cc, there is a function GetCurrentManifestPath. The goal of this task is to refactor ListColumnFamilies function so that ListColumnFamilies calls GetCurrentManifestPath to search for MANIFEST. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5331 Differential Revision: D15444524 Pulled By: HaoyuHuang fbshipit-source-id: 1dcbd030bc0f2e835695741f450bba150f2f2903
-
- 22 5月, 2019 2 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Remove PATENTS related wording from a few stragglers which still reference the old PATENTS file. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5326 Differential Revision: D15423297 Pulled By: sagar0 fbshipit-source-id: 4babcddfc120b7d2fed6eb3898287cf8012bf8ea
-
由 Siying Dong 提交于
Summary: Right now, in log writer, we call flush after writing each physical record. I don't see the necessarity of it. Right now, the underlying writer has a buffer, so there isn't a concern that the write request is too large either. On the other hand, in an Env where every flush is expensive, the current approach is significantly slower than only flushing after a whole record finishes, when the record is very large. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5328 Differential Revision: D15425032 Pulled By: siying fbshipit-source-id: 440ebef002dfbb60c59d8388c9ddfc83d79700aa
-
- 21 5月, 2019 2 次提交
-
-
由 Siying Dong 提交于
Summary: DBTablePropertiesTest.GetPropertiesOfTablesInRange sometimes hits the assert that generated LSM-tree doesn't have L1 file. Tighten the compaction triggering condition even further, hoping it goes away. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5302 Differential Revision: D15325971 Pulled By: siying fbshipit-source-id: 3e032bdb16fe8d98d5fcfcd65dd8be9781f3d6ae
-
由 Vijay Nadimpalli 提交于
Summary: Currently RocksDB uses Status::ShutdownInProgress to inform about column family drop. I would like to have a separate Status code for this event. https://github.com/facebook/rocksdb/blob/master/include/rocksdb/status.h#L55 Comment on this: https://github.com/facebook/rocksdb/blob/abc4202e47eb433dc731911af38f232d2148428c/db/version_set.cc#L2742:L2743 Pull Request resolved: https://github.com/facebook/rocksdb/pull/5275 Differential Revision: D15204583 Pulled By: vjnadimpalli fbshipit-source-id: 95e99e34b27bc165b554ecb8a48a7f8e60f21e2a
-
- 20 5月, 2019 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: WritePrepared transactions when configured with two_write_queues=true offers higher throughput with unordered_write feature without however compromising the rocksdb guarantees. This is because it performs ordering among writes in a 2nd step that is not tied to memtable write speed. The 2nd step is naturally provided by 2PC when the commit phase does the ordering as well. Without 2PC, the 2nd step would only be provided when we use two_write_queues=true, where WritePrepared after performing the writes, in a 2nd step uses the 2nd queue to assign order to the writes. The patch clarifies the need for two_write_queues=true in the HISTORY and inline comments of unordered_writes. Moreover it extends the stress tests of WritePrepared to unordred_write. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5313 Differential Revision: D15379977 Pulled By: maysamyabandeh fbshipit-source-id: 5b6f05b9b59285dcbf3b0532215ba9fe7d926e00
-
- 18 5月, 2019 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: RocksDB secondary can replay both MANIFEST and WAL now. On the one hand, the memory usage by memtables will grow after replaying WAL for sometime. On the other hand, replaying the MANIFEST can bring the database persistent data to a more recent point in time, giving us the opportunity to discard some memtables containing out-dated data. This PR coordinates the MANIFEST and WAL replay, using the updates from MANIFEST replay to update the active memtable and immutable memtable list of each column family. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5305 Differential Revision: D15386512 Pulled By: riversand963 fbshipit-source-id: a3ea6fc415f8382d8cf624f52a71ebdcffa3e355
-