- 01 8月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2670 Differential Revision: D5529018 Pulled By: siying fbshipit-source-id: 10a378933d509035d2dbe502247dd85fcea09789
-
- 29 7月, 2017 6 次提交
-
-
由 Siying Dong 提交于
Summary: Replace dynamic_cast<> so that users can choose to build with RTTI off, so that they can save several bytes per object, and get tiny more memory available. Some nontrivial changes: 1. Add Comparator::GetRootComparator() to get around the internal comparator hack 2. Add the two experiemental functions to DB 3. Add TableFactory::GetOptionString() to avoid unnecessary casting to get the option string 4. Since 3 is done, move the parsing option functions for table factory to table factory files too, to be symmetric. Closes https://github.com/facebook/rocksdb/pull/2645 Differential Revision: D5502723 Pulled By: siying fbshipit-source-id: fd13cec5601cf68a554d87bfcf056f2ffa5fbf7c
-
由 Mike Kolupaev 提交于
Summary: This fixes OOMs that we (logdevice) are currently having in production. SkipListRep constructor does a couple small allocations from ConcurrentArena (see InlineSkipList constructor). ConcurrentArena would sometimes allocate an entire block for that, which is a few megabytes (we use Options::arena_block_size = 4 MB). So an empty memtable can take take 4 MB of memory. We have ~40k column families (spread across 15 DB instances), so 4 MB per empty memtable easily OOMs a machine for us. This PR makes ConcurrentArena always allocate from Arena's inline block when possible. So as long as InlineSkipList's initial allocations are below 2 KB there would be no blocks allocated for empty memtables. Closes https://github.com/facebook/rocksdb/pull/2569 Differential Revision: D5404029 Pulled By: al13n321 fbshipit-source-id: 568ec22a3fd1a485c06123f6b2dfc5e9ef67cd23
-
由 Sagar Vemuri 提交于
Summary: - FIFOCompactionWithTTLTest was flaky when run in parallel earlier, and hence it was disabled. Fixed it now. - Also, faking sleep now instead of really sleeping to make tests more realistic by using TTLs like 1 hour and 1 day. Closes https://github.com/facebook/rocksdb/pull/2650 Differential Revision: D5506038 Pulled By: sagar0 fbshipit-source-id: deb429a527f045e3e2c5138b547c3e8ac8586aa2
-
由 Yi Wu 提交于
Summary: Move blob_db/ttl_extractor.h into blob_db/blob_db.h Also exclude TTLExtractor from LITE build. Closes https://github.com/facebook/rocksdb/pull/2665 Differential Revision: D5520009 Pulled By: yiwu-arbug fbshipit-source-id: 4813dcc272c7cc4bf2cdac285256d9a17d78c7b7
-
由 Sagar Vemuri 提交于
Summary: I might have missed these while doing some recent cassandra code reviews. Closes https://github.com/facebook/rocksdb/pull/2663 Differential Revision: D5520138 Pulled By: sagar0 fbshipit-source-id: 340930afe9efe03c75f535a1da1f89bd3e53c1f9
-
由 Islam AbdelRahman 提交于
Summary: Simple component that will add a new entry in a log file every time we lookup/insert a key in SimCache. API: ``` SimCache::StartActivityLogging(<file_name>, <env>, <optional_max_size>) SimCache::StopActivityLogging() ``` Sending for review, Still need to add more comments. I was thinking about a better approach, but I ended up deciding I will use a mutex to sync the writes to the file, since this feature should not be heavily used and only used to collect info that will be analyzed offline. I think it's okay to hold the mutex every time we lookup/add to the SimCache. Closes https://github.com/facebook/rocksdb/pull/2295 Differential Revision: D5063826 Pulled By: IslamAbdelRahman fbshipit-source-id: f3b5daed8b201987c9a071146ddd5c5740a2dd8c
-
- 28 7月, 2017 8 次提交
-
-
由 Yi Wu 提交于
Summary: Introducing blob_db::TTLExtractor to replace extract_ttl_fn. The TTL extractor can be use to extract TTL from keys insert with Put or WriteBatch. Change over existing extract_ttl_fn are: * If value is changed, it will be return via std::string* (rather than Slice*). With Slice* the new value has to be part of the existing value. With std::string* the limitation is removed. * It can optionally return TTL or expiration. Other changes in this PR: * replace `std::chrono::system_clock` with `Env::NowMicros` so that I can mock time in tests. * add several TTL tests. * other minor naming change. Closes https://github.com/facebook/rocksdb/pull/2659 Differential Revision: D5512627 Pulled By: yiwu-arbug fbshipit-source-id: 0dfcb00d74d060b8534c6130c808e4d5d0a54440
-
由 Andrew Kryczka 提交于
Summary: Breaking commit: d12691b8 In the above commit, I moved the `TableCache` cleanup logic from `Version` destructor into `PurgeObsoleteFiles`. I missed cleaning up `TableCache` entries for the current `Version` during DB destruction. This PR adds that logic to `VersionSet` destructor. One unfortunate side effect is now we're potentially deleting `TableReader`s after `column_family_set_.reset()`, which means we can't call `BlockBasedTableReader::Close` a second time as the block cache might already be destroyed. Closes https://github.com/facebook/rocksdb/pull/2662 Differential Revision: D5515108 Pulled By: ajkr fbshipit-source-id: 2cb820e19aa813e0d258d17f76b2d7b6b7ee0b18
-
由 Yi Wu 提交于
Summary: We don't need to set them explicitly. Closes https://github.com/facebook/rocksdb/pull/2660 Differential Revision: D5514141 Pulled By: yiwu-arbug fbshipit-source-id: 10edebfc3cfe0afc00a34519f87fcea4d65069ae
-
由 Siying Dong 提交于
Summary: platform_dependent tests in Travis now builds all tests, which is not needed. Only build those tests we need to run. Closes https://github.com/facebook/rocksdb/pull/2647 Differential Revision: D5513954 Pulled By: siying fbshipit-source-id: 4d540b146124e70dd25586c47939d19f93655b0a
-
由 Aaron Gao 提交于
Summary: solved https://github.com/facebook/rocksdb/issues/2604 Closes https://github.com/facebook/rocksdb/pull/2648 Differential Revision: D5504875 Pulled By: lightmark fbshipit-source-id: c14bb62ccbdc9e7bda9cd914cae4ea0765d882ee
-
由 Siying Dong 提交于
Summary: This is to work around the problem of build error: util/threadpool_imp.o: file not recognized: File truncated Just to make the build go through. We should remove it later if we find the real long-term solution. Closes https://github.com/facebook/rocksdb/pull/2657 Differential Revision: D5511034 Pulled By: siying fbshipit-source-id: 229f024bd78ee96799017d4a89be74253058ec30
-
由 Andrew Kryczka 提交于
Summary: Post-compaction work holds onto db mutex for the longest time (found by tracing lock acquires/releases with LTTng and correlating timestamps with our info log). Further experimentation showed `TableCache::EraseHandle` is responsible for ~86% of time mutex is held. We can just release the handle outside the db mutex. Closes https://github.com/facebook/rocksdb/pull/2654 Differential Revision: D5507126 Pulled By: ajkr fbshipit-source-id: 703c01ddf2aea16bc0f9e33c08935d78aa6b781d
-
由 Andrew Kryczka 提交于
Summary: it should be a bool Closes https://github.com/facebook/rocksdb/pull/2653 Differential Revision: D5506148 Pulled By: ajkr fbshipit-source-id: f142f0f3aa8b678c68adef12e5ac6e1e163306f3
-
- 27 7月, 2017 5 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2649 Differential Revision: D5505778 Pulled By: siying fbshipit-source-id: 7e935603ede3d958ea087ed6b8cfc4121e8797bc
-
由 Andres Suarez 提交于
Reviewed By: mzlee Differential Revision: D5472936 fbshipit-source-id: cf75858f879f1192b468a9020005634a5afad880
-
由 Siying Dong 提交于
Summary: This reverts commit 2289d381. Closes https://github.com/facebook/rocksdb/pull/2652 Differential Revision: D5506163 Pulled By: siying fbshipit-source-id: 105e31dd9d99090453a6b9f32c165206cd3affa3
-
由 Sagar Vemuri 提交于
Summary: A FIFO compaction picker test is accidentally testing against an instance of level compaction picker. Closes https://github.com/facebook/rocksdb/pull/2641 Differential Revision: D5495390 Pulled By: sagar0 fbshipit-source-id: 301962736f629b1c499570fb504cdbe66bacb46f
-
由 Kamalalochana Subbaiah 提交于
Summary: Support for PowerPC Architecture Detecting AltiVec Support Closes https://github.com/facebook/rocksdb/pull/2353 Differential Revision: D5210948 Pulled By: siying fbshipit-source-id: 859a8c063d37697addd89ba2b8a14e5efd5d24bf
-
- 26 7月, 2017 6 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: We initially had disabled support for write_options.sync when concurrent_prepare_ is set. We later added this support but the statement that asserts this combination is not used was left there. This patch cleans it up. Closes https://github.com/facebook/rocksdb/pull/2642 Differential Revision: D5496101 Pulled By: maysamyabandeh fbshipit-source-id: becbc503446f2a51bee24cc861958c090c724ec2
-
由 Yi Wu 提交于
Summary: The test is failing occasionally on the assert: `ASSERT_TRUE(writer->state == WriteThread::State::STATE_INIT)`. This is because the test don't make the leader wait for long enough before updating state for its followers. The patch move the update to `threads_waiting` to the end of `WriteThread::JoinBatchGroup:Wait` callback to avoid this happening. Also adding `WriteThread::JoinBatchGroup:Start` and have each thread wait there while another thread is linking to the linked-list. This is to make the check of `is_leader` more deterministic. Also changing two while-loops of `compare_exchange_strong` to plain `fetch_add`, to make it look cleaner. Closes https://github.com/facebook/rocksdb/pull/2640 Differential Revision: D5491525 Pulled By: yiwu-arbug fbshipit-source-id: 6e897f122082bd6f98e6d51b31a25e5fd0a3fb82
-
由 Yi Wu 提交于
Summary: 5.6.1 release blog post Closes https://github.com/facebook/rocksdb/pull/2638 Differential Revision: D5491168 Pulled By: yiwu-arbug fbshipit-source-id: 14e3a92a03684afa4bd19bfb3ffb053cc09f5d4a
-
由 Andrew Gallagher 提交于
Summary: These are implied by default platform flags, in particular, `-march=corei7`. Reviewed By: pixelb Differential Revision: D5485414 fbshipit-source-id: 85f1329c71fa81a604760844187cc73877fb40e9
-
由 Maysam Yabandeh 提交于
Summary: Currently this test times out with tsan. This is likely due to decreased speed with tsan. By lowering the number of iterations we can still catch a bug as the test is run regularly and multiple runs of the test is equivalent with running the test with more iterations. Closes https://github.com/facebook/rocksdb/pull/2639 Differential Revision: D5490549 Pulled By: maysamyabandeh fbshipit-source-id: bd69c42a9728d337ac95a06a401088384e51731a
-
由 Maysam Yabandeh 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2637 Differential Revision: D5489751 Pulled By: maysamyabandeh fbshipit-source-id: 0298f8960d4f86ce67959616615beee4d802c2e4
-
- 25 7月, 2017 10 次提交
-
-
由 Andres Suarez 提交于
Differential Revision: D5470179 fbshipit-source-id: 475ad3b6a85ead5f8abddefddc607353ea8680af
-
由 Thi Doan 提交于
Summary: Fixes broken links to the introductory talk I stumbled upon while reading the documentation. Closes https://github.com/facebook/rocksdb/pull/2628 Differential Revision: D5483851 Pulled By: sagar0 fbshipit-source-id: 94aab7fb4c4ed2305680a2fbc65b14c7977af6b8
-
由 atkawa7 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2629 Differential Revision: D5483751 Pulled By: sagar0 fbshipit-source-id: 9719ef9edd936dbb89b8988e3f4cb912a234f00e
-
由 Islam AbdelRahman 提交于
Summary: We will divide by zero if `stats.micros` is zero, just add a simple check This happens sometimes during running tests and UBSAN complains Closes https://github.com/facebook/rocksdb/pull/2631 Differential Revision: D5481455 Pulled By: IslamAbdelRahman fbshipit-source-id: 69aa24e64e21de15d9e2b8009adf01675fcc6598
-
由 kapitan-k 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2613 Differential Revision: D5476064 Pulled By: sagar0 fbshipit-source-id: 6b30a9eacb93a945bbe499eafb90565fa9f1798b
-
由 Daniel Black 提交于
Summary: I haven't looked to see if a class variable inside a loop like this is always initialised. Closes https://github.com/facebook/rocksdb/pull/2602 Differential Revision: D5475937 Pulled By: IslamAbdelRahman fbshipit-source-id: 8570b308f9a4b49e2a56ccc9e9b84d7c46568c15
-
由 Islam AbdelRahman 提交于
Summary: As explained in the comments, Sometimes we create Slice(nullptr, 0) in our code base which cause us to do calls like ``` memcmp(nullptr, "abc", 0); ``` That's fine since the len is equal 0, but UBSAN is not happy about it so disable UBSAN for this function and add an assert instead Closes https://github.com/facebook/rocksdb/pull/2616 Differential Revision: D5458326 Pulled By: IslamAbdelRahman fbshipit-source-id: cfca32abe30f7d8f760c9f77ecd9543dfb1170dd
-
由 Daniel Black 提交于
Summary: combining #2568 and #2612. Closes https://github.com/facebook/rocksdb/pull/2620 Differential Revision: D5464394 Pulled By: IslamAbdelRahman fbshipit-source-id: 9f71d3058dd6adaf02ce3b2de3a81a1228009778
-
由 Islam AbdelRahman 提交于
Summary: simply enable the macro in internal build, it wont hurt other sanitizers and will fix UBSAN issues Closes https://github.com/facebook/rocksdb/pull/2625 Differential Revision: D5475897 Pulled By: IslamAbdelRahman fbshipit-source-id: 262c6fd5de3c1906f4b29e55b39110f125f41057
-
由 Siying Dong 提交于
Summary: Add and implement Iterator::Refresh(). When this function is called, if the super version doesn't change, update the sequence number of the iterator to the latest one and invalidate the iterator. If the super version changed, recreated the whole iterator. This can help users reuse the iterator more easily. Closes https://github.com/facebook/rocksdb/pull/2621 Differential Revision: D5464500 Pulled By: siying fbshipit-source-id: f548bd35e85c1efca2ea69273802f6704eba6ba9
-
- 22 7月, 2017 4 次提交
-
-
由 Andrew Kryczka 提交于
Summary: The previous implementation of caching `file_size` index made no sense. It only remembered the original span of locked files starting from beginning of `file_size`. We should remember the index after all compactions that have been considered but rejected. This will reduce the work we do while holding the db mutex. Closes https://github.com/facebook/rocksdb/pull/2624 Differential Revision: D5468152 Pulled By: ajkr fbshipit-source-id: ab92a4bffe76f9f174d861bb5812b974d1013400
-
由 Sagar Vemuri 提交于
Summary: This reverts the previous commit 1d7048c5, which broke the build. Did a `git revert 1d7048c5`. Closes https://github.com/facebook/rocksdb/pull/2627 Differential Revision: D5476473 Pulled By: sagar0 fbshipit-source-id: 4756ff5c0dfc88c17eceb00e02c36176de728d06
-
由 Victor Gao 提交于
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually. Reviewed By: igorsugak Differential Revision: D5454343 fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
-
由 Pengchao Wang 提交于
Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
-