- 14 6月, 2017 6 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Added a flag, `ignore_unknown_options`, to skip unknown options when loading an options file (using `LoadLatestOptions`/`LoadOptionsFromFile`) or while verifying options (using `CheckOptionsCompatibility`). This will help in downgrading the db to an older version. Also added `--ignore_unknown_options` flag to ldb **Example Use case:** In MyRocks, if copying from newer version to older version, it is often impossible to start because of new RocksDB options that don't exist in older version, even though data format is compatible. MyRocks uses these load and verify functions in [ha_rocksdb.cc::check_rocksdb_options_compatibility](https://github.com/facebook/mysql-5.6/blob/e004fd9f416821d043ccc8ad4a345c33ac9953f0/storage/rocksdb/ha_rocksdb.cc#L3348-L3401). **Test Plan:** Updated the unit tests. `make check` ldb: $ ./ldb --db=/tmp/test_db --create_if_missing put a1 b1 OK Now edit /tmp/test_db/<OPTIONS-file> and add an unknown option. Try loading the options now, and it fails: $ ./ldb --db=/tmp/test_db --try_load_options get a1 Failed: Invalid argument: Unrecognized option DBOptions:: abcd Passes with the new --ignore_unknown_options flag $ ./ldb --db=/tmp/test_db --try_load_options --ignore_unknown_options get a1 b1 Closes https://github.com/facebook/rocksdb/pull/2423 Differential Revision: D5212091 Pulled By: sagar0 fbshipit-source-id: 2ec17636feb47dc0351b53a77e5f15ef7cbf2ca7
-
由 hyunwoo 提交于
Summary: fixed typo Closes https://github.com/facebook/rocksdb/pull/2430 Differential Revision: D5242471 Pulled By: IslamAbdelRahman fbshipit-source-id: 832eb3a4c70221444ccd2ae63217823fec56c748
-
由 haoxiang 提交于
Summary: fixed a typo in util/dynamic_bloom.h Closes https://github.com/facebook/rocksdb/pull/2442 Differential Revision: D5242397 Pulled By: IslamAbdelRahman fbshipit-source-id: c47fd18cc79afff6b022201a0410c0cd47626576
-
由 Andrew Kryczka 提交于
Summary: Allow users to rate limit background work based on read bytes, written bytes, or sum of read and written bytes. Support these by changing the RateLimiter API, so no additional options were needed. Closes https://github.com/facebook/rocksdb/pull/2433 Differential Revision: D5216946 Pulled By: ajkr fbshipit-source-id: aec57a8357dbb4bfde2003261094d786d94f724e
-
由 Yi Wu 提交于
Summary: At the beginning of write batch write, grab the latest sequence from base db and assume sequence number will increment by 1 for each put and delete, and write the exact sequence number with each put. This is assuming we are the only writer to increment sequence number (no external file ingestion, etc) and there should be no holes in the sequence number. Also having some minor naming changes. Closes https://github.com/facebook/rocksdb/pull/2402 Differential Revision: D5176134 Pulled By: yiwu-arbug fbshipit-source-id: cb4712ee44478d5a2e5951213a10b72f08fe8c88
-
由 Maysam Yabandeh 提交于
Summary: When Partitioning index/filter is enabled the user might need to check the index block size as well as the top-level index size via sst_dump. This patch records i) number of partitions, ii) top-level index size and make it accessible through sst_dump. The number of partitions for filters is the same as that of indexes. The top-level index for filters has a similar size to top-level index for indexes, so it is not repeated. Closes https://github.com/facebook/rocksdb/pull/2437 Differential Revision: D5224225 Pulled By: maysamyabandeh fbshipit-source-id: 5324598c75793523aef1bb7ee225a5475e95a9cb
-
- 13 6月, 2017 7 次提交
-
-
由 Siying Dong 提交于
Summary: 5582123d broken CLANG release build because of an unexpected change. Fix it. Closes https://github.com/facebook/rocksdb/pull/2443 Differential Revision: D5236297 Pulled By: siying fbshipit-source-id: 1b410adf13ded149c53e8235e9ea9f3130fb5403
-
由 Siying Dong 提交于
Summary: Currently for direct I/O, the large maximum buffer is always allocated. This will be wasteful if users flush the data in much smaller chunks. This diff fix this by changing the behavior of incremental buffer works. When we enlarge buffer, we try to copy the existing data in the buffer to the enlarged buffer, rather than flush the buffer first. This can make sure that no extra I/O is introduced because of buffer enlargement. Closes https://github.com/facebook/rocksdb/pull/2403 Differential Revision: D5178403 Pulled By: siying fbshipit-source-id: a8fe1e7304bdb8cab2973340022fe80ff83449fd
-
由 Anirban Rahut 提交于
Summary: initialize 2 additional fields tm_gmtoff and tm_zone, otherwise under strict warnings for initialization, we get errors in myrocks. Closes https://github.com/facebook/rocksdb/pull/2439 Differential Revision: D5229013 Pulled By: yiwu-arbug fbshipit-source-id: 9fc1615a1919656f36064791706ed41e10e9db84
-
由 Islam AbdelRahman 提交于
Summary: Update DeleteScheduler to delete files immediately if trash directory is >= 25% of DB size Closes https://github.com/facebook/rocksdb/pull/2436 Differential Revision: D5230384 Pulled By: IslamAbdelRahman fbshipit-source-id: 5cbda8ac536a3cc72c774641621edc02c8202482
-
由 Orgad Shaneh 提交于
Summary: cc tamird sagar0 Closes https://github.com/facebook/rocksdb/pull/2325 Differential Revision: D5098302 Pulled By: sagar0 fbshipit-source-id: 297c5506b5d9b2ed1d7719c8caf0b96cffe503b8
-
由 Sagar Vemuri 提交于
Summary: This diff syncs the Histogram Types in the Java API with the ones in C++ API (`statistics.h`), and brings it up-to-date. I also found that the enum ordering between Java and C++ has gotten out-of-sync, a few years back, with the addition of `SUBCOMPACTION_SETUP_TIME`. So updated the order as well. `READ_NUM_MERGE_OPERANDS` added in #2373 is needed for Cassandra-on-RocksDB work. Closes https://github.com/facebook/rocksdb/pull/2429 Differential Revision: D5215623 Pulled By: sagar0 fbshipit-source-id: bd136698c48197e53693275eb52acc9198ee5a4e
-
由 Andrew Kryczka 提交于
Summary: - mention range deletion + file ingestion - move post-5.6 stuff into new section Closes https://github.com/facebook/rocksdb/pull/2440 Differential Revision: D5229910 Pulled By: ajkr fbshipit-source-id: 1facfe41993fa1f3b1f6fa7dc77d2b11aa2b317a
-
- 12 6月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: We estimate number of reads per SST files, by updating the counter per file in sampled read requests. This information can later be used to trigger compactions to improve read performacne. Closes https://github.com/facebook/rocksdb/pull/2417 Differential Revision: D5193528 Pulled By: siying fbshipit-source-id: b4241c5ad0eaf444b61afb53f8e6290d9f5da2df
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2419 Differential Revision: D5193976 Pulled By: siying fbshipit-source-id: 62d115edee6043237e9d6ad3c2a05481e162c9eb
-
- 10 6月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: update bug fixed. Closes https://github.com/facebook/rocksdb/pull/2434 Differential Revision: D5218601 Pulled By: lightmark fbshipit-source-id: 1f86b2c93345673612381081537d464e7d12e434
-
- 09 6月, 2017 5 次提交
-
-
由 Mike Kolupaev 提交于
Summary: We had a crash in this code: `fstat()` failed; `file_stats` contained garbage, in particular `file_stats.st_blksize == 6`; the expression `file_stats.st_blocks / (file_stats.st_blksize / 512)` divided by zero. Closes https://github.com/facebook/rocksdb/pull/2420 Differential Revision: D5216110 Pulled By: al13n321 fbshipit-source-id: 6d8fc5e7c4f98c1139e68c7829ebdbac68b0fce0
-
由 Yi Wu 提交于
Summary: Mingw is complaining about uninitialized variable in mock_env.cc. e.g. https://travis-ci.org/facebook/rocksdb/jobs/240132276 The fix is to initialize the variable. Closes https://github.com/facebook/rocksdb/pull/2428 Differential Revision: D5211306 Pulled By: yiwu-arbug fbshipit-source-id: ee02bf0327dcea8590a2aa087f0176fecaf8621c
-
由 Sagar Vemuri 提交于
Summary: Adding SSTFileWriter's newly introduced put, merge and delete apis to the Java api. The C++ APIs were first introduced in #2361. Add is deprecated in favor of Put. Merge is especially needed to support streaming for Cassandra-on-RocksDB work in https://issues.apache.org/jira/browse/CASSANDRA-13476. Closes https://github.com/facebook/rocksdb/pull/2392 Differential Revision: D5165091 Pulled By: sagar0 fbshipit-source-id: 6f0ad396a7cbd2e27ca63e702584784dd72acaab
-
由 Yi Wu 提交于
Summary: UBSAN crashes when it run the test. Disabling it for UBSAN. Closes https://github.com/facebook/rocksdb/pull/2427 Differential Revision: D5210897 Pulled By: yiwu-arbug fbshipit-source-id: 2f5a876807c98d8db79ab9581965f7e6b29d4163
-
由 Maysam Yabandeh 提交于
Summary: /home/travis/build/facebook/rocksdb/env/mock_env.cc: In member function ‘virtual void rocksdb::{anonymous}::TestMemLogger::Logv(const char*, va_list)’: /home/travis/build/facebook/rocksdb/env/mock_env.cc:391:53: error: ‘t.tm::tm_year’ may be used uninitialized in this function [-Werror=maybe-uninitialized] static_cast<int>(now_tv.tv_usec)); Closes https://github.com/facebook/rocksdb/pull/2418 Differential Revision: D5193597 Pulled By: maysamyabandeh fbshipit-source-id: 8801a3ef27f33eb419d534f7de747702cdf504a0
-
- 07 6月, 2017 2 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: USE_CLANG=1 make -j32 analyze The two errors would disappear after the assertion. Closes https://github.com/facebook/rocksdb/pull/2416 Differential Revision: D5193526 Pulled By: maysamyabandeh fbshipit-source-id: 16a21f18f68023f862764dd3ab9e00ca60b0eefa
-
由 Maysam Yabandeh 提交于
Summary: filter_block_set_ access must also be protected with mutex. Closes https://github.com/facebook/rocksdb/pull/2413 Differential Revision: D5193159 Pulled By: maysamyabandeh fbshipit-source-id: 6987fc219d9a65c20b9c7e52151aef4b8e4882e6
-
- 06 6月, 2017 7 次提交
-
-
由 Aaron Gao 提交于
Summary: Bump version to 5.6 beforehand in master Closes https://github.com/facebook/rocksdb/pull/2411 Differential Revision: D5186896 Pulled By: lightmark fbshipit-source-id: 079538e621b1a959c2dc99dada894e9cdb99ef95
-
由 Yi Wu 提交于
Summary: Attempt to force travis to build with clang on MacOS Closes https://github.com/facebook/rocksdb/pull/2408 Differential Revision: D5186635 Pulled By: yiwu-arbug fbshipit-source-id: dbb779eff07b1cb7dbd2092631303cf946316656
-
由 Sagar Vemuri 提交于
Summary: There are a couple of warnings while building RocksJava, coming from Javadoc generation. ``` Generating target/apidocs/org/rocksdb/RocksDB.html... src/main/java/org/rocksdb/RocksDB.java:2139: warning: no throws for org.rocksdb.RocksDBException public void ingestExternalFile(final List<String> filePathList, ^ src/main/java/org/rocksdb/RocksDB.java:2162: warning: no throws for org.rocksdb.RocksDBException public void ingestExternalFile(final ColumnFamilyHandle columnFamilyHandle, ^ ``` Closes https://github.com/facebook/rocksdb/pull/2396 Differential Revision: D5178388 Pulled By: sagar0 fbshipit-source-id: a0ab6696d6de78d089a9a860a559f64cc320019e
-
由 Siying Dong 提交于
Summary: If ReadOptions.low_pri=true and compaction is behind, the write will either return immediate or be slowed down based on ReadOptions.no_slowdown. Closes https://github.com/facebook/rocksdb/pull/2369 Differential Revision: D5127619 Pulled By: siying fbshipit-source-id: d30e1cff515890af0eff32dfb869d2e4c9545eb0
-
由 Adam Retter 提交于
Summary: Updates the statically linked libraries from linking against glibc 2.5, to linking against glibc 2.12. Closes https://github.com/facebook/rocksdb/pull/2405 Differential Revision: D5184132 Pulled By: sagar0 fbshipit-source-id: 7a8ad4cf7e737ca62f29e58938bd49fa02114541
-
由 Yi Wu 提交于
Summary: Fix db_write_test clang/windows build failure. Explicitly cast size_t (unsigned long) to uint32_t (unsigned int). Closes https://github.com/facebook/rocksdb/pull/2407 Differential Revision: D5182995 Pulled By: yiwu-arbug fbshipit-source-id: aba225a9fccb12d5bfbdc2cd6efc11040706a9d2
-
由 hyunwoo 提交于
Summary: fixed typo Closes https://github.com/facebook/rocksdb/pull/2376 Differential Revision: D5183630 Pulled By: ajkr fbshipit-source-id: 133cfd0445959e70aa2cd1a12151bf3c0c5c3ac5
-
- 05 6月, 2017 1 次提交
-
-
由 Adam Retter 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2406 Differential Revision: D5181091 Pulled By: ajkr fbshipit-source-id: fd72525da4fb1d50143080a210f8d824cbb968d6
-
- 03 6月, 2017 6 次提交
-
-
由 Aaron Gao 提交于
Summary: clean up the current test dir if the last regression test is still running. Closes https://github.com/facebook/rocksdb/pull/2401 Differential Revision: D5177882 Pulled By: lightmark fbshipit-source-id: 91d899fcc2bde841948eae71af8584d4bdb35468
-
由 Aaron Gao 提交于
Summary: … headers https://github.com/facebook/rocksdb/pull/2199 should not reference RocksDB-specific macros (like ROCKSDB_SUPPORT_THREAD_LOCAL in this case) to public headers, `iostats_context.h` and `perf_context.h`. We shouldn't do that because users have to provide these compiler flags when building their binary with RocksDB. We should hide the thread local global variable inside our implementation and just expose a function api to retrieve these variables. It may break some users for now but good for long term. make check -j64 Closes https://github.com/facebook/rocksdb/pull/2380 Differential Revision: D5177896 Pulled By: lightmark fbshipit-source-id: 6fcdfac57f2e2dcfe60992b7385c5403f6dcb390
-
由 Mike Kolupaev 提交于
Summary: Fixes the following scenario: 1. Set prefix extractor. Enable bloom filters, with `whole_key_filtering = false`. Use compaction filter that sometimes returns `kRemoveAndSkipUntil`. 2. Do a compaction. 3. Compaction creates an iterator with `total_order_seek = false`, calls `SeekToFirst()` on it, then repeatedly calls `Next()`. 4. At some point compaction filter returns `kRemoveAndSkipUntil`. 5. Compaction calls `Seek(skip_until)` on the iterator. The key that it seeks to happens to have prefix that doesn't match the bloom filter. Since `total_order_seek = false`, iterator becomes invalid, and compaction thinks that it has reached the end. The rest of the compaction input is silently discarded. The fix is to make compaction iterator use `total_order_seek = true`. The implementation for PlainTable is quite awkward. I've made `kRemoveAndSkipUntil` officially incompatible with PlainTable. If you try to use them together, compaction will fail, and DB will enter read-only mode (`bg_error_`). That's not a very graceful way to communicate a misconfiguration, but the alternatives don't seem worth the implementation time and complexity. To be able to check in advance that `kRemoveAndSkipUntil` is not going to be used with PlainTable, we'd need to extend the interface of either `CompactionFilter` or `InternalIterator`. It seems unlikely that anyone will ever want to use `kRemoveAndSkipUntil` with PlainTable: PlainTable probably has very few users, and `kRemoveAndSkipUntil` has only one user so far: us (logdevice). Closes https://github.com/facebook/rocksdb/pull/2349 Differential Revision: D5110388 Pulled By: lightmark fbshipit-source-id: ec29101a99d9dcd97db33923b87f72bce56cc17a
-
由 Siying Dong 提交于
Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
-
由 Andrew Kryczka 提交于
Summary: Some users want to monitor column family activity in their custom memtable implementations. Previously there was no way to figure out with which column family a memtable is associated. This diff: - adds an overload to MemTableRepFactory::CreateMemTableRep() that provides the CF ID. For compatibility, its default implementation calls the old overload. - updates MemTable to create MemTableRep's using the new overload. Closes https://github.com/facebook/rocksdb/pull/2346 Differential Revision: D5108061 Pulled By: ajkr fbshipit-source-id: 3a1921214a348dd8ea0f54e1cab3b71c3d46d616
-
由 Yi Wu 提交于
Summary: rocksdb::Random is not thread-safe. Have one Random for each thread instead. Closes https://github.com/facebook/rocksdb/pull/2400 Differential Revision: D5173919 Pulled By: yiwu-arbug fbshipit-source-id: 1a99c7b877f3893eb22355af49e321bcad4e53e6
-
- 02 6月, 2017 3 次提交
-
-
由 Andrew Kryczka 提交于
Summary: The range deletion meta-block iterators weren't getting cleaned up properly since they don't support arena allocation. I didn't implement arena support since, in the general case, each iterator is used only once and separately from all other iterators, so there should be no benefit to data locality. Anyways, this diff fixes up #2370 by treating range deletion iterators as non-arena-allocated. Closes https://github.com/facebook/rocksdb/pull/2399 Differential Revision: D5171119 Pulled By: ajkr fbshipit-source-id: bef6f5c4c5905a124f4993945aed4bd86e2807d8
-
由 Andrew Kryczka 提交于
Summary: also changed the `>` in the comparison against `level0_file_num_compaction_trigger` into a `>=` since exactly `level0_file_num_compaction_trigger` can trigger a compaction from L0. Closes https://github.com/facebook/rocksdb/pull/2179 Differential Revision: D4915772 Pulled By: ajkr fbshipit-source-id: e38fec6253de6f9a40e61734615c6670d84038aa
-
由 Andrew Gallagher 提交于
Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
-