- 05 5月, 2017 4 次提交
-
-
由 Siying Dong 提交于
Summary: Allow an option for users to do some compaction in FIFO compaction, to pay some write amplification for fewer number of files. Closes https://github.com/facebook/rocksdb/pull/2163 Differential Revision: D4895953 Pulled By: siying fbshipit-source-id: a1ab608dd0627211f3e1f588a2e97159646e1231
-
由 Andrew Kryczka 提交于
Summary: Changed dynamic leveling to stop setting the base level's size bound below `max_bytes_for_level_base`. Behavior for config where `max_bytes_for_level_base == level0_file_num_compaction_trigger * write_buffer_size` and same amount of data in L0 and base-level: - Before #2027, compaction scoring would favor base-level due to dividing by size smaller than `max_bytes_for_level_base`. - After #2027, L0 and Lbase get equal scores. The disadvantage is L0 is often compacted before reaching the num files trigger since `write_buffer_size` can be bigger than the dynamically chosen base-level size. This increases write-amp. - After this diff, L0 and Lbase still get equal scores. Now it takes `level0_file_num_compaction_trigger` files of size `write_buffer_size` to trigger L0 compaction by size, fixing the write-amp problem above. Closes https://github.com/facebook/rocksdb/pull/2123 Differential Revision: D4861570 Pulled By: ajkr fbshipit-source-id: 467ddef56ed1f647c14d86bb018bcb044c39b964
-
由 Andrew Kryczka 提交于
Summary: When user doesn't set a limit on compaction output file size, let's use the sum of the input files' sizes. This will avoid passing UINT64_MAX as fallocate()'s length. Reported in #2249. Test setup: - command: `TEST_TMPDIR=/data/rocksdb-test/ strace -e fallocate ./db_compaction_test --gtest_filter=DBCompactionTest.ManualCompactionUnknownOutputSize` - filesystem: xfs before this diff: `fallocate(10, 01, 0, 1844674407370955160) = -1 ENOSPC (No space left on device)` after this diff: `fallocate(10, 01, 0, 1977) = 0` Closes https://github.com/facebook/rocksdb/pull/2252 Differential Revision: D5007275 Pulled By: ajkr fbshipit-source-id: 4491404a6ae8a41328aede2e2d6f4d9ac3e38880
-
由 Leonidas Galanis 提交于
Summary: Followup to make 0x40000 a TableCache constant that indicates infinite capacity Closes https://github.com/facebook/rocksdb/pull/2247 Differential Revision: D5001349 Pulled By: lgalanis fbshipit-source-id: ce7bd2e54b0975bb9f8680fdaa0f8bb0e7ae81a2
-
- 04 5月, 2017 4 次提交
-
-
由 Aaron Gao 提交于
Summary: we align the buffer with logical sector size and should not test it with page size, which is usually 4k. Closes https://github.com/facebook/rocksdb/pull/2245 Differential Revision: D5001842 Pulled By: lightmark fbshipit-source-id: a7135fcf6351c6db363e8908956b1e193a4a6291
-
由 Leonidas Galanis 提交于
Summary: Makes max_open_files db option dynamically set-able by SetDBOptions. During the call of SetDBOptions we call SetCapacity on the table cache, which is a LRUCache. Closes https://github.com/facebook/rocksdb/pull/2185 Differential Revision: D4979189 Pulled By: yiwu-arbug fbshipit-source-id: ca7e8dc5e3619c79434f579be4847c0f7e56afda
-
由 Jos Collin 提交于
Summary: Checked the return value of __get_cpuid(). Implemented the else case where the arch is different from i386 and x86_64. Pulled By: ajkr Differential Revision: D4973496 fbshipit-source-id: c40fdef5840364c2a79b1d11df0db5d4ec3d6a4a
-
由 siddontang 提交于
Summary: Try to fix https://github.com/facebook/rocksdb/issues/1969 Closes https://github.com/facebook/rocksdb/pull/2170 Differential Revision: D4907333 Pulled By: yiwu-arbug fbshipit-source-id: 417b420ff668e6c2fd0dad42a94c57385012edc5
-
- 03 5月, 2017 4 次提交
-
-
由 Adam Retter 提交于
Summary: Remove orphaned Java classes that were missed from commit 9385fd72 Closes https://github.com/facebook/rocksdb/issues/2210 Closes https://github.com/facebook/rocksdb/pull/2212 Differential Revision: D4986199 Pulled By: maysamyabandeh fbshipit-source-id: 74e866cd8661c172a4754c2c945e30d650f6fa05
-
由 Tamir Duberstein 提交于
Summary: siying Closes https://github.com/facebook/rocksdb/pull/2233 Differential Revision: D4987113 Pulled By: siying fbshipit-source-id: f07ecd3f7d9f5366a2b1665ce28ba10d74405557
-
由 Siying Dong 提交于
Summary: A data race between a manual and an auto compaction can cause a scheduled automatic compaction to be cancelled and never rescheduled again. This may cause a condition of hanging forever. Fix this by always making sure the cancelled compaction is put back to the compaction queue. Closes https://github.com/facebook/rocksdb/pull/2238 Differential Revision: D4984591 Pulled By: siying fbshipit-source-id: 3ab153886403c7b991896dcb2158b96cac12f227
-
由 Maysam Yabandeh 提交于
Summary: This reverts commit 89833577. Closes https://github.com/facebook/rocksdb/pull/2240 Differential Revision: D4986982 Pulled By: maysamyabandeh fbshipit-source-id: 56c4c07b7b5b7c6fe122d5c2f2199d221c8510c0
-
- 02 5月, 2017 4 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: Some filters such as partitioned filter have pointers to the table for which they are created. Therefore is they are stored in the block cache, the should be forcibly erased from block cache before closing the table, which would result into deleting the object. Otherwise the destructor will be called later when the cache is lazily erasing the object, which having the parent table no longer existent it could result into undefined behavior. Update: there will be still cases the filter is not removed from the cache since the table has not kept a pointer to the cache handle to be able to forcibly release it later. We make sure that the filter destructor does not access the table pointer to get around such cases. Closes https://github.com/facebook/rocksdb/pull/2207 Differential Revision: D4941591 Pulled By: maysamyabandeh fbshipit-source-id: 56fbab2a11cf447e1aa67caa30b58d7bd7ce5bbd
-
由 Maysam Yabandeh 提交于
Summary: With row cache being enabled, table cache is doing a short circuit for reading data. This path needs to be updated to take advantage of pinnable slice. In the meanwhile we disabling pinning in this path. Closes https://github.com/facebook/rocksdb/pull/2237 Differential Revision: D4982389 Pulled By: maysamyabandeh fbshipit-source-id: 542630d0cf23cfb1f0c397da82e7053df7966591
-
由 Siying Dong 提交于
Summary: ColumnFamilyData::ConstructNewMemtable is called out of DB mutex, and it asserts current_ is not empty, but current_ should only be accessed inside DB mutex. Remove this assert to make TSAN happy. Closes https://github.com/facebook/rocksdb/pull/2235 Differential Revision: D4978531 Pulled By: siying fbshipit-source-id: 423685a7dae88ed3faaa9e1b9ccb3427ac704a4b
-
由 Siying Dong 提交于
Summary: VALGRIND_VER was left empty after moving the environment to GCC-5. Set it back. Closes https://github.com/facebook/rocksdb/pull/2234 Differential Revision: D4978534 Pulled By: siying fbshipit-source-id: f0640d58e8f575f75fb3f8b92e686c9e0b6a59bb
-
- 29 4月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: TSAN sometimes complaints data race of PosixLogger::flush_pending_. Make it atomic. Closes https://github.com/facebook/rocksdb/pull/2231 Differential Revision: D4973397 Pulled By: siying fbshipit-source-id: 571e886e3eca3231705919d573e250c1c1ec3764
-
由 Yi Wu 提交于
Summary: Fix use after scope error caught by ASAN. Closes https://github.com/facebook/rocksdb/pull/2228 Differential Revision: D4968028 Pulled By: yiwu-arbug fbshipit-source-id: a2a266c98634237494ab4fb2d666bc938127aeb2
-
- 28 4月, 2017 4 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2226 Differential Revision: D4967547 Pulled By: siying fbshipit-source-id: dd3b58ae1e7a106ab6bb6f37ab5c88575b125ab4
-
由 Joel Marcey 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2224 Differential Revision: D4966524 Pulled By: siying fbshipit-source-id: 99f28471bcfc83b0a9df871d5247f62d4a6540ce
-
由 Aaron Gao 提交于
Summary: In case users cast a subclass of db* into dbimpl* Closes https://github.com/facebook/rocksdb/pull/2222 Differential Revision: D4964486 Pulled By: lightmark fbshipit-source-id: 0ccdc08ee8e7a193dfbbe0218c3cbfd795662ca1
-
由 Dmitri Smirnov 提交于
Summary: Remove double buffering on RandomRead on Windows. With more logic appear in file reader/write Read no longer obeys forwarding calls to Windows implementation. Previously direct_io (unbuffered) was only available on Windows but now is supported as generic. We remove intermediate buffering on Windows. Remove random_access_max_buffer_size option which was windows specific. Non-zero values for that opton introduced unnecessary lock contention. Remove Env::EnableReadAhead(), Env::ShouldForwardRawRequest() that are no longer necessary. Add aligned buffer reads for cases when requested reads exceed read ahead size. Closes https://github.com/facebook/rocksdb/pull/2105 Differential Revision: D4847770 Pulled By: siying fbshipit-source-id: 8ab48f8e854ab498a4fd398a6934859792a2788f
-
- 27 4月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2217 Differential Revision: D4959351 Pulled By: siying fbshipit-source-id: ce7c3a430bae0d15e06b3d5c958ebce969d08564
-
由 Aaron Gao 提交于
Summary: �fix the buffer size in case of ppl use buffer size as their block_size. Closes https://github.com/facebook/rocksdb/pull/2198 Differential Revision: D4956878 Pulled By: lightmark fbshipit-source-id: 8bb0dc9c133887aadcd625d5261a3d1110b71473
-
由 Andrew Kryczka 提交于
Summary: It resets all the ticker and histogram stats to zero. Needed to change the locking a bit since Reset() is the only operation that manipulates multiple tickers/histograms together, and that operation should be seen as atomic by other operations that access tickers/histograms. Closes https://github.com/facebook/rocksdb/pull/2213 Differential Revision: D4952232 Pulled By: ajkr fbshipit-source-id: c0475c3e4c7b940120d53891b69c3091149a0679
-
由 Aaron Gao 提交于
Summary: Every time after a compaction/flush finish, we issue user reads to put the table into block cache which includes a couple of IO that read footer, index blocks, meta block, etc. So we implement Prefetch here to reduce IO. Closes https://github.com/facebook/rocksdb/pull/2196 Differential Revision: D4931782 Pulled By: lightmark fbshipit-source-id: 5a13d58dcab209964352322217193bbf7ff78149
-
由 Andrew Kryczka 提交于
Summary: I'm going to add more DB tests for statistics as currently we have very few. I started a file dedicated to this purpose and moved the existing stats-specific tests there. Closes https://github.com/facebook/rocksdb/pull/2211 Differential Revision: D4951558 Pulled By: ajkr fbshipit-source-id: 05d11c35079c40ecabdfd2cf5556ccb761f694a4
-
由 Aaron Gao 提交于
Summary: Support buck load with universal compaction. More test cases to be added. Closes https://github.com/facebook/rocksdb/pull/2202 Differential Revision: D4935360 Pulled By: lightmark fbshipit-source-id: cc3ca1b6f42faa503207dab1408d6bcf393ee5b5
-
- 26 4月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: https://github.com/facebook/rocksdb/issues/2152 Closes https://github.com/facebook/rocksdb/pull/2208 Differential Revision: D4945577 Pulled By: lightmark fbshipit-source-id: 4e679150f2c9443d3be0b6008b26b65fabbda75a
-
- 25 4月, 2017 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: These code paths forked when checkpoint was introduced by copy/pasting the core backup logic. Over time they diverged and bug fixes were sometimes applied to one but not the other (like fix to include all relevant WALs for 2PC), or it required extra effort to fix both (like fix to forge CURRENT file). This diff reunites the code paths by extracting the core logic into a function, CreateCustomCheckpoint(), that is customizable via callbacks to implement both checkpoint and backup. Related changes: - flush_before_backup is now forcibly enabled when 2PC is enabled - Extracted CheckpointImpl class definition into a header file. This is so the function, CreateCustomCheckpoint(), can be called by internal rocksdb code but not exposed to users. - Implemented more functions in DummyDB/DummyLogFile (in backupable_db_test.cc) that are used by CreateCustomCheckpoint(). Closes https://github.com/facebook/rocksdb/pull/1932 Differential Revision: D4622986 Pulled By: ajkr fbshipit-source-id: 157723884236ee3999a682673b64f7457a7a0d87
-
由 Aaron Gao 提交于
Summary: User could call this with wrapper class of DB or DBImpl Closes https://github.com/facebook/rocksdb/pull/2200 Differential Revision: D4935530 Pulled By: lightmark fbshipit-source-id: df9cb61d67d0f3bbcf62f714d77523a459a92883
-
由 Maysam Yabandeh 提交于
Summary: This is useful when we put the entries in the block cache for accounting purposes and do not expect it to be used after it is released. If the cache does not erase the item in such cases not only the performance of cache is negatively affected but the item's destructor not being called at the time of release might violate the assumptions about the lifetime of the object. The new change adds a force_erase option to the Release method and returns a boolean to indicate whehter the item is successfully deleted. Closes https://github.com/facebook/rocksdb/pull/2180 Differential Revision: D4916032 Pulled By: maysamyabandeh fbshipit-source-id: 94409a346069923cac9de8e57adc313b4ed46f28
-
- 22 4月, 2017 3 次提交
-
-
由 Tomas Kolda 提交于
Summary: Replacement of #2147 The change was squashed due to a lot of conflicts. Closes https://github.com/facebook/rocksdb/pull/2194 Differential Revision: D4929799 Pulled By: siying fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581
-
由 Maysam Yabandeh 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2197 Differential Revision: D4932434 Pulled By: maysamyabandeh fbshipit-source-id: 6c83c12d6d47e3f0640ab84954944215968f266f
-
由 Nick Anderson 提交于
Summary: In the off chance you have a `Microsoft.PowerShell_profile.ps1` profile the `execute_process` will load one's profile during running `cmake`, which will cause garbage output to be written to the `build_version.cc` file. If you add in a `-noprofile` flag, or you suppress output in your pshell profile, this is mitigated. Closes https://github.com/facebook/rocksdb/pull/2168 Differential Revision: D4927003 Pulled By: siying fbshipit-source-id: 83861752d6cf2627dd864eedd2acaa8aa8a6232e
-
- 21 4月, 2017 5 次提交
-
-
由 Aaron Gao 提交于
Summary: It is a potential bug that will be triggered if we ingest files before inserting the first key into an empty db. 0 is a special value reserved to indicate the concept of non-existence. But not good for seqno in this case because 0 is a valid seqno for ingestion(bulk loading) Closes https://github.com/facebook/rocksdb/pull/2183 Differential Revision: D4919827 Pulled By: lightmark fbshipit-source-id: 237eea40f88bd6487b66806109d90065dc02c362
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2190 Differential Revision: D4922688 Pulled By: siying fbshipit-source-id: eaa3d122f8d389ae0508ec8b61f7780fd8b0a7ef
-
由 Tomas Kolda 提交于
Summary: Workaround for Solaris gcc binary. Program is crashing, because when TLS of perf context that is used twice on same frame, it is damaged thus Segmentation fault. Issue: #2153 Closes https://github.com/facebook/rocksdb/pull/2187 Differential Revision: D4922274 Pulled By: siying fbshipit-source-id: 549105ebce9a8ce08a737f4d6b9f2312ebcde9a8
-
由 Siying Dong 提交于
Summary: In a previous commit, I changed the way to checkout release branches from "git checkout <branch_name>" to "git checkout origin/<branch_name>". However, this doesn't seem to work in our CI environment. Revert it. Closes https://github.com/facebook/rocksdb/pull/2189 Differential Revision: D4922294 Pulled By: siying fbshipit-source-id: 482c17f9b05e6ccb190876b050682fe5a458103d
-
由 Siying Dong 提交于
Summary: tools/check_format_compatible.sh will check a newer version of RocksDB can open option files generated by older version releases. In order to achieve that, a new parameter "--try_load_options" is added to ldb. With this parameter set, if option file exists, we load the option file and use it to open the DB. With this opiton set, we can validate option loading logic. Closes https://github.com/facebook/rocksdb/pull/2178 Differential Revision: D4914989 Pulled By: siying fbshipit-source-id: db114f7724fcb41e5e9483116d84d7c4b8389ca4
-