- 03 6月, 2017 1 次提交
-
-
由 Yi Wu 提交于
Summary: rocksdb::Random is not thread-safe. Have one Random for each thread instead. Closes https://github.com/facebook/rocksdb/pull/2400 Differential Revision: D5173919 Pulled By: yiwu-arbug fbshipit-source-id: 1a99c7b877f3893eb22355af49e321bcad4e53e6
-
- 02 6月, 2017 8 次提交
-
-
由 Andrew Kryczka 提交于
Summary: The range deletion meta-block iterators weren't getting cleaned up properly since they don't support arena allocation. I didn't implement arena support since, in the general case, each iterator is used only once and separately from all other iterators, so there should be no benefit to data locality. Anyways, this diff fixes up #2370 by treating range deletion iterators as non-arena-allocated. Closes https://github.com/facebook/rocksdb/pull/2399 Differential Revision: D5171119 Pulled By: ajkr fbshipit-source-id: bef6f5c4c5905a124f4993945aed4bd86e2807d8
-
由 Andrew Kryczka 提交于
Summary: also changed the `>` in the comparison against `level0_file_num_compaction_trigger` into a `>=` since exactly `level0_file_num_compaction_trigger` can trigger a compaction from L0. Closes https://github.com/facebook/rocksdb/pull/2179 Differential Revision: D4915772 Pulled By: ajkr fbshipit-source-id: e38fec6253de6f9a40e61734615c6670d84038aa
-
由 Andrew Gallagher 提交于
Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
-
由 Aaron Gao 提交于
Summary: For level_compaction_style regression test. Closes https://github.com/facebook/rocksdb/pull/2397 Differential Revision: D5168545 Pulled By: lightmark fbshipit-source-id: 195e4d84917e7c261d9f4fbe9aee5d104c9cb9a2
-
https://github.com/facebook/rocksdb/pull/2082由 Maysam Yabandeh 提交于
Summary: This is a manual commit of this PR: Retire InMemoryEnv in favor of MockEnv #2082 With MockEnv doing the same yet being more mature, InMemoryEnv is redundant. Reviewed By: IslamAbdelRahman Differential Revision: D5162323 fbshipit-source-id: 59fd0082a891dc99cc531e4da9d68bf891eae3f5
-
由 Islam AbdelRahman 提交于
-
由 Volker Mische 提交于
Summary: When the `TwoLevelIndexSearch` was introduced, it wasn't added to the C-API. Closes https://github.com/facebook/rocksdb/pull/2395 Differential Revision: D5165127 Pulled By: maysamyabandeh fbshipit-source-id: d077f16ab5646c18158d8202a33b0fd076c6c8ad
-
由 Daniel Black 提交于
Summary: Use later xcode version from https://docs.travis-ci.com/user/osx-ci-environment Closes https://github.com/facebook/rocksdb/pull/2128 Differential Revision: D4907471 Pulled By: yiwu-arbug fbshipit-source-id: debf8e27baef71a5833c845401b1865bc75ac977
-
- 01 6月, 2017 4 次提交
-
-
由 Tamir Duberstein 提交于
Summary: This fixes a compilation failure on Linux when the system libc is not glibc. jemalloc's configure script incorrectly assumes that glibc is always used on Linux systems, producing glibc-style signatures; when the system libc is e.g. musl, the following error is observed: ``` [ 0%] Building CXX object CMakeFiles/rocksdb.dir/db/db_impl.cc.o In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/table/block.h:19:0, from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:77: /x-tools/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot/usr/include/malloc.h:19:8: error: declaration of 'size_t malloc_usable_size(void*)' has a different exception specifier size_t malloc_usable_size(void *); ^~~~~~~~~~~~~~~~~~ In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:20:0: /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:78:33: note: from previous declaration 'size_t malloc_usable_size(void*) throw ()' # define je_malloc_usable_size malloc_usable_size ^ /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:239:41: note: in expansion of macro 'je_malloc_usable_size' JEMALLOC_EXPORT size_t JEMALLOC_NOTHROW je_malloc_usable_size( ^~~~~~~~~~~~~~~~~~~~~ CMakeFiles/rocksdb.dir/build.make:350: recipe for target 'CMakeFiles/rocksdb.dir/db/db_impl.cc.o' failed ``` This works around the issue by rearranging the sources such that jemalloc's headers are never in the same scope as the system's malloc header. The jemalloc issue has been reported as well, see: https://github.com/jemalloc/jemalloc/issues/778. cc tschottdorf Closes https://github.com/facebook/rocksdb/pull/2188 Differential Revision: D5163048 Pulled By: siying fbshipit-source-id: c553125458892def175c1be5682b0330d80b2a0d
-
由 Aaron Gao 提交于
Summary: fix regression test by not reporting stats when building db Closes https://github.com/facebook/rocksdb/pull/2390 Differential Revision: D5159909 Pulled By: lightmark fbshipit-source-id: c3f4b9deb9c6799ff84207fd341c529144f8158d
-
由 Andrew Kryczka 提交于
Summary: Previously we returned NotSupported when ingesting files into a database containing any range deletions. This diff adds the support. - Flush if any memtable contains range deletions overlapping the to-be-ingested file - Place to-be-ingested file before any level that contains range deletions overlapping it. - Added support for `Version` to return iterators over range deletions in a given level. Previously, we piggybacked getting range deletions onto `Version`'s `Get()` / `AddIterator()` functions by passing them a `RangeDelAggregator*`. But file ingestion needs to get iterators over range deletions, not populate an aggregator (since the aggregator does collapsing and doesn't expose the actual ranges). Closes https://github.com/facebook/rocksdb/pull/2370 Differential Revision: D5127648 Pulled By: ajkr fbshipit-source-id: 816faeb9708adfa5287962bafdde717db56e3f1a
-
由 Yi Wu 提交于
Summary: Blob db rely on base db returning sequence number through write batch after DB::Write(). However after recent changes to the write path, DB::Writ()e no longer return sequence number in some cases. Fixing it by have WriteBatchInternal::InsertInto() always encode sequence number into write batch. Stacking on #2375. Closes https://github.com/facebook/rocksdb/pull/2385 Differential Revision: D5148358 Pulled By: yiwu-arbug fbshipit-source-id: 8bda0aa07b9334ed03ed381548b39d167dc20c33
-
- 31 5月, 2017 5 次提交
-
-
由 Siying Dong 提交于
Summary: Add a histogram in statistics to help users understand how many merge operands they merge. Closes https://github.com/facebook/rocksdb/pull/2373 Differential Revision: D5139983 Pulled By: siying fbshipit-source-id: 61b9ba8ca83f358530a4833d68f0103b56a0e182
-
由 Yi Wu 提交于
Summary: Re-enable blob_db_test with some update: * Commented out delay at the end of GC tests. Will update the logic later with sync point to properly trigger GC. * Added some helper functions. Also update make files to include blob_dump tool. Closes https://github.com/facebook/rocksdb/pull/2375 Differential Revision: D5133793 Pulled By: yiwu-arbug fbshipit-source-id: 95470b26d0c1f9592ba4b7637e027fdd263f425c
-
由 Aaron Gao 提交于
Summary: abandon fillseqdeterministic test locally Closes https://github.com/facebook/rocksdb/pull/2290 Differential Revision: D5151867 Pulled By: lightmark fbshipit-source-id: 4c8a24cc937212ffb5ceb9bfaf7288eb8726d0c1
-
由 Tamir Duberstein 提交于
Summary: yiwu-arbug see individual commits. Closes https://github.com/facebook/rocksdb/pull/2318 Differential Revision: D5141520 Pulled By: yiwu-arbug fbshipit-source-id: 7987c92ab4461eef36afce5a133d3a0ee0c96300
-
由 Tamir Duberstein 提交于
Summary: This collapses all the "platform dependent" tests into a single travis builder in an effort to reduce overall CI times. These builds currently take a combined 21-23 minutes, but each one has to compile the library, so combining them should yield some time savings (5-10 minutes). Unfortunately the other builders don't duplicate work, so combining them is unlikely to provide benefit. Closes https://github.com/facebook/rocksdb/pull/2306 Differential Revision: D5147850 Pulled By: yiwu-arbug fbshipit-source-id: d947dc8b9f49639fe22f3c8ab9a82a8d730ddddf
-
- 27 5月, 2017 7 次提交
-
-
由 Sagar Vemuri 提交于
Summary: 5.4.5 release blog post. Tag created at: https://github.com/facebook/rocksdb/releases/tag/rocksdb-5.4.5 Closes https://github.com/facebook/rocksdb/pull/2378 Differential Revision: D5141648 Pulled By: sagar0 fbshipit-source-id: 43faadfbd7feb8bb01c3d56127ef3453e23dc28a
-
由 Sagar Vemuri 提交于
Summary: Adding my name to the authors list so that I can publish a post to rocksdb blog (rocksdb.org). Closes https://github.com/facebook/rocksdb/pull/2379 Differential Revision: D5143582 Pulled By: sagar0 fbshipit-source-id: d85163f8b59aaeb07ac2a1cdd776ae335c7062b9
-
由 赵星宇 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2377 Differential Revision: D5141274 Pulled By: lightmark fbshipit-source-id: c237a285b73ad93488c080ea80c71a29a17f1be0
-
由 Aaron Gao 提交于
Summary: Previously sst_file_writer only supports kTypeValue, we need kTypeMerge and kTypeDeletion also as user requested. Closes https://github.com/facebook/rocksdb/pull/2361 Differential Revision: D5139402 Pulled By: lightmark fbshipit-source-id: 092a60756d01692539d817a3765ebfd58a8d7f88
-
由 Siying Dong 提交于
Summary: Reorder variables of ReadOptions so that its size is reduced from 64 to 48 bytes. Closes https://github.com/facebook/rocksdb/pull/2366 Differential Revision: D5124043 Pulled By: siying fbshipit-source-id: 70e9c204c34f97fad011f2fe2297ba292d85df7a
-
由 Sagar Vemuri 提交于
Summary: This log message shouldn't be a warning; some services are seeing high warning count due to this. The count for the below line is a few hundreds of millions, as per Logview: ``` [rocksdb/src/db/column_family.cc:729] [checkpoints] Increasing compaction threads because we have 2 level-0 files ``` Closes https://github.com/facebook/rocksdb/pull/2364 Differential Revision: D5123565 Pulled By: sagar0 fbshipit-source-id: a07ce499a4f82f0ebde9cda9f4948fb9df6a734c
-
由 Sagar Vemuri 提交于
Summary: zstd files are downloaded and used as part of JNI build, but are left behind even after doing a `make clean`. This PR updates the `clean` target to remove these zstd files as well. Closes https://github.com/facebook/rocksdb/pull/2365 Differential Revision: D5123537 Pulled By: sagar0 fbshipit-source-id: a8f355da5ba961aa89d5852e35751ffc35de03ea
-
- 26 5月, 2017 3 次提交
-
-
由 Andrew Gallagher 提交于
Reviewed By: meyering Differential Revision: D5094332 fbshipit-source-id: 3df2f693def8ca418bc9febe3e20ccf051f2e19d
-
由 Yi Wu 提交于
Summary: Lite build does not recognize FLAGS_use_blob_db. Fixing it. Closes https://github.com/facebook/rocksdb/pull/2372 Reviewed By: anirbanr-fb Differential Revision: D5130773 Pulled By: yiwu-arbug fbshipit-source-id: 43131d9d0be5811f2129af562be72cca26369cb3
-
由 Aaron Gao 提交于
Summary: rand_ has data race risk TEST_TMPDIR=\/dev\/shm\/rocksdb OPT=-g COMPILE_WITH_TSAN=1 CRASH_TEST_KILL_ODD=1887 make J=1 crash_test Closes https://github.com/facebook/rocksdb/pull/2368 Differential Revision: D5127424 Pulled By: lightmark fbshipit-source-id: b7f4d1430a5769b57da9f99037106749264b2ced
-
- 25 5月, 2017 7 次提交
-
-
由 Andrew Kryczka 提交于
Summary: stop calling Close() at the end of tests holding a compaction pressure token since it causes the write controller to be deleted while it's still needed. these calls were pointless anyways since Close() is already called in the test's destructor. Closes https://github.com/facebook/rocksdb/pull/2367 Differential Revision: D5125906 Pulled By: ajkr fbshipit-source-id: 6cad8673e5546a82ff602ac0ba59cc3f68dbde46
-
由 Daniel Black 提交于
Summary: just realised when I updated the .travis.yml to trusty the llvm repo was still precise. Update this and clang-4.0. Closes https://github.com/facebook/rocksdb/pull/2127 Differential Revision: D4869427 Pulled By: sagar0 fbshipit-source-id: b7f906b6fac28e60cacc6a1f1959d6acf8269906
-
由 Aaron Gao 提交于
Summary: update targets file for release Closes https://github.com/facebook/rocksdb/pull/2358 Differential Revision: D5115705 Pulled By: lightmark fbshipit-source-id: 96a3c7e15b5807b5d0f5a9bb73850b92754b5794
-
由 Andrew Kryczka 提交于
Summary: - `max_background_flushes` and `max_background_compactions` are still supported for backwards compatibility - `base_background_compactions` is completely deprecated. Now we just throttle to one background compaction when there's no pressure. - `max_background_jobs` is added to automatically partition the concurrent background jobs into flushes vs compactions. Currently it's very simple as we just allocate one-fourth of the jobs to flushes, and the remaining can be used for compactions. - The test cases that set `base_background_compactions > 1` needed to be updated. I just grab the pressure token such that the desired number of compactions can be scheduled. Closes https://github.com/facebook/rocksdb/pull/2205 Differential Revision: D4937461 Pulled By: ajkr fbshipit-source-id: df52cbbd497e13bbc9a60560a5ac2a2526b3f1f9
-
由 Adam Retter 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2356 Differential Revision: D5122240 Pulled By: sagar0 fbshipit-source-id: 3764797dd261d5b2a04d58b3967969e66c541bbb
-
由 Siying Dong 提交于
Summary: It's hard for RocksDB to come up with a good default of delayed write rate. Use rate given by rate limiter if it is availalbe. This provides the I/O order of magnitude. Closes https://github.com/facebook/rocksdb/pull/2357 Differential Revision: D5115324 Pulled By: siying fbshipit-source-id: 341065ad2211c981fc804011c0f0e59a50c7e754
-
由 Siying Dong 提交于
Summary: We forgot to add the new flag in internal build script. Add it. Closes https://github.com/facebook/rocksdb/pull/2360 Differential Revision: D5121428 Pulled By: siying fbshipit-source-id: af72d48cd855b37df1ce3c1fbb00c80377ba6e4f
-
- 24 5月, 2017 5 次提交
-
-
由 Sagar Vemuri 提交于
Summary: Fixing the build errors seen with GCC 4.8.1. ``` Makefile:105: Warning: Compiling in debug mode. Don't use the resulting binary in production utilities/blob_db/blob_dump_tool.cc: In member function ‘rocksdb::Status rocksdb::blob_db::BlobDumpTool::DumpBlobLogFooter(uint64_t, uint64_t*)’: utilities/blob_db/blob_dump_tool.cc:149:42: error: expected ‘)’ before ‘PRIu64’ fprintf(stdout, " Blob count : %" PRIu64 "\n", footer.GetBlobCount()); ^ utilities/blob_db/blob_dump_tool.cc:149:76: error: spurious trailing ‘%’ in format [-Werror=format=] fprintf(stdout, " Blob count : %" PRIu64 "\n", footer.GetBlobCount()); ^ utilities/blob_db/blob_dump_tool.cc:149:76: error: too many arguments for format [-Werror=format-extra-args] utilities/blob_db/blob_dump_tool.cc: In member function ‘rocksdb::Status rocksdb::blob_db::BlobDumpTool::DumpRecord(rocksdb::blob_db::BlobDumpTool::DisplayType, rocksdb::blob_db::BlobDumpTool::DisplayType, uint64_t*)’: utilities/blob_db/blob_dump_tool.cc:161:49: error: expected ‘)’ before ‘PRIx64’ fprintf(stdout, "Read record with offset 0x%" PRIx64 " (%" PRIu64 "):\n", ^ utilities/blob_db/blob_dump_tool.cc:162:27: error: spurious trailing ‘%’ in format [-Werror=format=] *offset, *offset); ^ utilities/blob_db/blob_dump_tool.cc:162:27: error: too many arguments for format [-Werror=format-extra-args] utilities/blob_db/blob_dump_tool.cc:176:38: error: expected ‘)’ before ‘PRIu64’ fprintf(stdout, " blob size : %" PRIu64 "\n", record.GetBlobSize()); ^ utilities/blob_db/blob_dump_tool.cc:176:71: error: spurious trailing ‘%’ in format [-Werror=format=] fprintf(stdout, " blob size : %" PRIu64 "\n", record.GetBlobSize()); ^ utilities/blob_db/blob_dump_tool.cc:176:71: error: too many arguments for format [-Werror=format-extra-args] utilities/blob_db/blob_dump_tool.cc:178:38: error: expected ‘)’ before ‘PRIu64’ fprintf(stdout, " time : %" PRIu64 "\n", record.GetTimeVal()); ^ utilities/blob_db/blob_dump_tool.cc:178:70: error: spurious trailing ‘%’ in format [-Werror=format=] fprintf(stdout, " time : %" PRIu64 "\n", record.GetTimeVal()); ^ utilities/blob_db/blob_dump_tool.cc:178:70: error: too many arguments for format [-Werror=format-extra-args] utilities/blob_db/blob_dump_tool.cc:214:38: error: expected ‘)’ before ‘PRIu64’ fprintf(stdout, " sequence : %" PRIu64 "\n", record.GetSN()); ^ utilities/blob_db/blob_dump_tool.cc:214:65: error: spurious trailing ‘%’ in format [-Werror=format=] fprintf(stdout, " sequence : %" PRIu64 "\n", record.GetSN()); ``` Closes https://github.com/facebook/rocksdb/pull/2359 Differential Revision: D5117684 Pulled By: sagar0 fbshipit-source-id: 7480346bcd96205fcae890927c5e68cf004e87be
-
由 Igor Canadi 提交于
Summary: `cf_stats_snapshot_.seconds_up` appears to be never updated, unlike `db_stats_snapshot_.seconds_up`, which is updated here: https://github.com/facebook/rocksdb/blob/master/db/internal_stats.cc#L883 This leads to wrong information in the log, for example: ``` ** Compaction Stats [default] ** .... Uptime(secs): 85591.2 total, 85591.2 interval ``` Even though DB's interval is correctly logged as 60 seconds: ``` ** DB Stats ** Uptime(secs): 85591.2 total, 637.8 interval ``` Closes https://github.com/facebook/rocksdb/pull/2338 Differential Revision: D5114131 Pulled By: sagar0 fbshipit-source-id: 85243a38213236ccbb601a7f7aaa8865eaa8083c
-
由 Sagar Vemuri 提交于
Summary: Fix build error in db_iter.cc when running clang-analyzer. ``` CC db/db_iter.o db/db_iter.cc:938:21: error: no matching constructor for initialization of 'rocksdb::ParsedInternalKey' ParsedInternalKey ikey(Slice(), 0, 0); ^ ~~~~~~~~~~~~~ ./db/dbformat.h:84:3: note: candidate constructor not viable: no known conversion from 'int' to 'rocksdb::ValueType' for 3rd argument ParsedInternalKey(const Slice& u, const SequenceNumber& seq, ValueType t) ^ ./db/dbformat.h:78:8: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 3 were provided struct ParsedInternalKey { ^ ./db/dbformat.h:78:8: note: candidate constructor (the implicit move constructor) not viable: requires 1 argument, but 3 were provided ./db/dbformat.h:83:3: note: candidate constructor not viable: requires 0 arguments, but 3 were provided ParsedInternalKey() { } // Intentionally left uninitialized (for speed) ^ 1 error generated. ``` Closes https://github.com/facebook/rocksdb/pull/2354 Differential Revision: D5115751 Pulled By: sagar0 fbshipit-source-id: b0e386d4e935e4725b07761c3ca5f7a8cbde3692
-
由 Sagar Vemuri 提交于
Summary: Release builds are failing on Linux with the error: ``` tools/db_stress.cc: In function ‘int main(int, char**)’: tools/db_stress.cc:2365:12: error: ‘rocksdb::SyncPoint’ has not been declared rocksdb::SyncPoint::GetInstance()->SetCallBack( ^ tools/db_stress.cc:2370:12: error: ‘rocksdb::SyncPoint’ has not been declared rocksdb::SyncPoint::GetInstance()->SetCallBack( ^ tools/db_stress.cc:2375:12: error: ‘rocksdb::SyncPoint’ has not been declared rocksdb::SyncPoint::GetInstance()->EnableProcessing(); ^ make[1]: *** [tools/db_stress.o] Error 1 make[1]: Leaving directory `/data/sandcastle/boxes/trunk-git-rocksdb-public' make: *** [release] Error 2 ``` Closes https://github.com/facebook/rocksdb/pull/2355 Differential Revision: D5113552 Pulled By: sagar0 fbshipit-source-id: 351df707277787da5633ba4a40e52edc7c895dc4
-
由 Giuseppe Ottaviano 提交于
Summary: The default IO priority of WritableFiles is IO_TOTAL, meaning that they will bypass the rate limiter if it's passed in the options. This change allows to pass an io priority in construction, so that by setting IO_LOW or IO_HIGH the rate limit will be honored. It also fixes a minor bug: SstFileWriter's copy and move constructor are not disabled and incorrect, as any copy/move will result in a double free. Switching to unique_ptr makes the object correctly movable and non-copyable as expected. Also fix minor style inconsistencies. Closes https://github.com/facebook/rocksdb/pull/2335 Differential Revision: D5113260 Pulled By: sagar0 fbshipit-source-id: e084236e7ff0b50a56cbeceaa9fedd5e210bf9f8
-