- 05 4月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: I've needed Env timing measurements a few times now, so finally built something for it. Closes https://github.com/facebook/rocksdb/pull/2073 Differential Revision: D4811231 Pulled By: ajkr fbshipit-source-id: 218a249
-
- 04 4月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: It is confusing to have auto_roll_logger to stay under db/, which has nothing to do with database. Move filename together as it is a dependency. Closes https://github.com/facebook/rocksdb/pull/2080 Differential Revision: D4821141 Pulled By: siying fbshipit-source-id: ca7d768
-
由 Tamir Duberstein 提交于
Summary: yuslepukhin See individual commits. Closes https://github.com/facebook/rocksdb/pull/2051 Differential Revision: D4819697 Pulled By: siying fbshipit-source-id: 75c1de4
-
- 31 3月, 2017 1 次提交
-
-
由 Orgad Shaneh 提交于
Summary: There still are many warnings (most of them about invalid printf format for long long), but it builds if FAIL_ON_WARNINGS is disabled. Closes https://github.com/facebook/rocksdb/pull/2052 Differential Revision: D4807355 Pulled By: siying fbshipit-source-id: ef03786
-
- 30 3月, 2017 1 次提交
-
-
由 Min Wei 提交于
Summary: Currently the fast crc32 path is not enabled on Windows. I am trying to enable it here, hopefully, with the minimum impact to the existing code structure. Closes https://github.com/facebook/rocksdb/pull/2033 Differential Revision: D4770635 Pulled By: siying fbshipit-source-id: 676f8b8
-
- 04 3月, 2017 1 次提交
-
-
由 Maysam Yabandeh 提交于
Summary: This is the first split of https://github.com/facebook/rocksdb/pull/1891 and will be needed for the upcoming partitioned filter patch. Closes https://github.com/facebook/rocksdb/pull/1949 Differential Revision: D4652152 Pulled By: maysamyabandeh fbshipit-source-id: 9801778
-
- 01 3月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Separate the platform dependent tests from external_sst_file_test. Only those tests need to run on platforms like OSX Closes https://github.com/facebook/rocksdb/pull/1923 Differential Revision: D4622461 Pulled By: siying fbshipit-source-id: d2d6f04
-
- 28 2月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Separate a smal subset of tests in DBTest to DBBasicTest. Tests in DBTest don't have to run in CI tests on platforms like OSX, as long as they are covered by Linux. Closes https://github.com/facebook/rocksdb/pull/1924 Differential Revision: D4616702 Pulled By: siying fbshipit-source-id: 13e6549
-
- 24 2月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Xfunc is hardly used. Remove it to keep the code simple. Closes https://github.com/facebook/rocksdb/pull/1905 Differential Revision: D4603220 Pulled By: siying fbshipit-source-id: 731f96d
-
- 07 2月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: introduce new methods into a public threadpool interface, - allow submission of std::functions as they allow greater flexibility. - add Joining methods to the implementation to join scheduled and submitted jobs with an option to cancel jobs that did not start executing. - Remove ugly `#ifdefs` between pthread and std implementation, make it uniform. - introduce pimpl for a drop in replacement of the implementation - Introduce rocksdb::port::Thread typedef which is a replacement for std::thread. On Posix Thread defaults as before std::thread. - Implement WindowsThread that allocates memory in a more controllable manner than windows std::thread with a replaceable implementation. - should be no functionality changes. Closes https://github.com/facebook/rocksdb/pull/1823 Differential Revision: D4492902 Pulled By: siying fbshipit-source-id: c74cb11
-
- 03 2月, 2017 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: merger.h was always a confusing name for me, simply give the file a better name Closes https://github.com/facebook/rocksdb/pull/1836 Differential Revision: D4505357 Pulled By: IslamAbdelRahman fbshipit-source-id: 07b28d8
-
- 26 1月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: The Env registration framework supports registering client Envs and selecting which one to instantiate according to a text field. This enabled things like adding the -env_uri argument to db_bench, so the same binary could be reused with different Envs just by changing CLI config. Now this problem has come up again in a non-Env context, as I want to instantiate a client Statistics implementation from db_bench, which is configured entirely via text parameters. Also, in the future we may wish to use it for deserializing client objects when loading OPTIONS file. This diff generalizes the Env registration logic to work with arbitrary types. - Generalized registration and instantiation code by templating them - The entire implementation is in a header file as that's Google style guide's recommendation for template definitions - Pattern match with std::regex_match rather than checking prefix, which was the previous behavior - Rename functions/files to be non-Env-specific Closes https://github.com/facebook/rocksdb/pull/1776 Differential Revision: D4421933 Pulled By: ajkr fbshipit-source-id: 34647d1
-
- 25 1月, 2017 1 次提交
-
-
由 Arun Sharma 提交于
Summary: After make install, I see a directory hierarchy that looks like ``` ./usr ./usr/include ./usr/include/rocksdb ./usr/include/rocksdb/filter_policy.h [..] ./usr/include/rocksdb/iterator.h ./usr/include/rocksdb/utilities ./usr/include/rocksdb/utilities/ldb_cmd_execute_result.h ./usr/include/rocksdb/utilities/lua ./usr/include/rocksdb/utilities/lua/rocks_lua_custom_library.h ./usr/include/rocksdb/utilities/lua/rocks_lua_util.h ./usr/include/rocksdb/utilities/lua/rocks_lua_compaction_filter.h ./usr/include/rocksdb/utilities/backupable_db.h [..] ./usr/include/rocksdb/utilities/env_registry.h [..] ./usr/include/rocksdb/env.h ./usr/lib64 ./usr/lib64/librocksdb.so.5 ./usr/lib64/librocksdb.so.5.0.0 ./usr/lib64/librocksdb.so ./usr/lib64/librocksdb.a ``` Closes https://github.com/facebook/rocksdb/pull/1798 Differential Revision: D4456536 Pulled By: yiwu-arbug fbshipit-source-id: 5494e91
-
- 21 1月, 2017 2 次提交
-
-
由 Arun Sharma 提交于
Summary: The libraries produced on linux are now named librocksdb.a librocksdb.so Other fixes: * Also link with -lrt to avoid linker errors. * Generalize comments at the top to include Linux * Move -lgtest before -lpthread to avoid linker errors * move add_subdirectory(tools) to the end so it picks up the right libraries Closes https://github.com/facebook/rocksdb/pull/1364 Differential Revision: D4444138 Pulled By: yiwu-arbug fbshipit-source-id: f0e2c19
-
由 Kefu Chai 提交于
Summary: because not all archs support this option. see https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html. also do not pass "-fno-omit-frame-pointer" and "-momit-leaf-frame-pointer" to compiler if ${CMAKE_BUILD_TYPE} is "Debug". this matches the behaviour of DEBUG_LEVEL=2 in Makefile. Signed-off-by: NKefu Chai <tchaikov@gmail.com> Closes https://github.com/facebook/rocksdb/pull/1762 Differential Revision: D4444036 Pulled By: yiwu-arbug fbshipit-source-id: 8596fbe
-
- 17 12月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Iterator should be in corrupted status if merge operator return false. Also add test to make sure if max_successive_merges is hit during write, data will not be lost. Closes https://github.com/facebook/rocksdb/pull/1665 Differential Revision: D4322695 Pulled By: yiwu-arbug fbshipit-source-id: b327b05
-
- 02 12月, 2016 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Now that we have userspace persisted cache, we don't need flashcache anymore. Closes https://github.com/facebook/rocksdb/pull/1588 Differential Revision: D4245114 Pulled By: igorcanadi fbshipit-source-id: e2c1c72
-
- 30 11月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/1578 Differential Revision: D4241171 Pulled By: ajkr fbshipit-source-id: ce5fd83
-
- 19 11月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
Summary: Signed-off-by: NKefu Chai <tchaikov@gmail.com> Closes https://github.com/facebook/rocksdb/pull/1540 Differential Revision: D4207564 Pulled By: siying fbshipit-source-id: 567415b
-
- 17 11月, 2016 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff includes an implementation of CompactionFilter that allows users to write CompactionFilter in Lua. With this ability, users can dynamically change compaction filter logic without requiring building the rocksdb binary and restarting the database. To compile, WITH_LUA_PATH must be specified to the base directory of lua. Closes https://github.com/facebook/rocksdb/pull/1478 Differential Revision: D4150138 Pulled By: yhchiang fbshipit-source-id: ed84222
-
- 16 11月, 2016 1 次提交
-
-
由 Willem Jan Withagen 提交于
Summary: This will allow reference to `malloc_stats_print` Closes https://github.com/facebook/rocksdb/pull/1516 Differential Revision: D4187258 Pulled By: siying fbshipit-source-id: 34ae9f9
-
- 14 11月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Currently our skip-list have an optimization to speedup sequential inserts from a single stream, by remembering the last insert position. We extend the idea to support sequential inserts from multiple streams, and even tolerate small reordering wihtin each stream. This PR is the interface part adding the following: - Add `memtable_insert_prefix_extractor` to allow specifying prefix for each key. - Add `InsertWithHint()` interface to memtable, to allow underlying implementation to return a hint of insert position, which can be later pass back to optimize inserts. - Memtable will maintain a map from prefix to hints and pass the hint via `InsertWithHint()` if `memtable_insert_prefix_extractor` is non-null. Closes https://github.com/facebook/rocksdb/pull/1419 Differential Revision: D4079367 Pulled By: yiwu-arbug fbshipit-source-id: 3555326
-
- 01 11月, 2016 1 次提交
-
-
由 Vladislav Vaintroub 提交于
Summary: If FAIL_ON_WARNINGS is OFF, compilation will succeed if there were compile warnings. Closes https://github.com/facebook/rocksdb/pull/1446 Differential Revision: D4107825 Pulled By: yiwu-arbug fbshipit-source-id: 4ce6e05
-
- 29 10月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
this breaks the cross-compiling, and we can not assume that the building machine and the target machine share the same CPU spec. Signed-off-by: NKefu Chai <kchai@redhat.com>
-
- 26 10月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
* util/build_verion.cc.in: add this file, so cmake and make can share the template file for generating util/build_version.cc. * CMakeLists.txt: also, cmake v2.8.11 does not support file(GENERATE ...), so we are using configure_file() for creating build_version.cc. * Makefile: use util/build_verion.cc.in for creating build_version.cc. Signed-off-by: NKefu Chai <tchaikov@gmail.com>
-
- 21 10月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Changes in the diff API changes: - Introduce IngestExternalFile to replace AddFile (I think this make the API more clear) - Introduce IngestExternalFileOptions (This struct will encapsulate the options for ingesting the external file) - Deprecate AddFile() API Logic changes: - If our file overlap with the memtable we will flush the memtable - We will find the first level in the LSM tree that our file key range overlap with the keys in it - We will find the lowest level in the LSM tree above the the level we found in step 2 that our file can fit in and ingest our file in it - We will assign a global sequence number to our new file - Remove AddFile restrictions by using global sequence numbers Other changes: - Refactor all AddFile logic to be encapsulated in ExternalSstFileIngestionJob Test Plan: unit tests (still need to add more) addfile_stress (https://reviews.facebook.net/D65037) Reviewers: yiwu, andrewkr, lightmark, yhchiang, sdong Reviewed By: sdong Subscribers: jkedgar, hcz, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D65061
-
- 19 10月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This diff introduces RangeDelAggregator, which takes ownership of iterators provided to it via AddTombstones(). The tombstones are organized in a two-level map (snapshot stripe -> begin key -> tombstone). Tombstone creation avoids data copy by holding Slices returned by the iterator, which remain valid thanks to pinning. For compaction, we create a hierarchical range tombstone iterator with structure matching the iterator over compaction input data. An aggregator based on that iterator is used by CompactionIterator to determine which keys are covered by range tombstones. In case of merge operand, the same aggregator is used by MergeHelper. Upon finishing each file in the compaction, relevant range tombstones are added to the output file's range tombstone metablock and file boundaries are updated accordingly. To check whether a key is covered by range tombstone, RangeDelAggregator::ShouldDelete() considers tombstones in the key's snapshot stripe. When this function is used outside of compaction, it also checks newer stripes, which can contain covering tombstones. Currently the intra-stripe check involves a linear scan; however, in the future we plan to collapse ranges within a stripe such that binary search can be used. RangeDelAggregator::AddToBuilder() adds all range tombstones in the table's key-range to a new table's range tombstone meta-block. Since range tombstones may fall in the gap between files, we may need to extend some files' key-ranges. The strategy is (1) first file extends as far left as possible and other files do not extend left, (2) all files extend right until either the start of the next file or the end of the last range tombstone in the gap, whichever comes first. One other notable change is adding release/move semantics to ScopedArenaIterator such that it can be used to transfer ownership of an arena-allocated iterator, similar to how unique_ptr is used for malloc'd data. Depends on D61473 Test Plan: compaction_iterator_test, mock_table, end-to-end tests in D63927 Reviewers: sdong, IslamAbdelRahman, wanning, yhchiang, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62205
-
- 13 10月, 2016 1 次提交
-
-
- 11 10月, 2016 1 次提交
-
-
由 Bassam Tabbara 提交于
Signed-off-by: NBassam Tabbara <bassam.tabbara@quantum.com>
-
- 29 9月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
* enable cmake to work on linux and osx also * port part of build_detect_platform not covered by thirdparty.inc to cmake. - detect fallocate() - detect malloc_usable_size() - detect JeMalloc - detect snappy * check for asan,tsan,ubsan * create 'build_version.cc' in build directory. * add `check` target to support 'make check'. * add `tools` target to match its counterpart in Makefile. * use `date` on non-win32 platforms. * pass different cflags on non-win32 platforms * detect pthead library using FindThread cmake module. * enable CMP0042 to silence the cmake warning on osx * reorder the linked libraries. because testutillib references gtest, to enable the linker to find the referenced symbols, we need to put gtest after testutillib. Signed-off-by: NMarcus Watts <mwatts@redhat.com> Signed-off-by: NKefu Chai <kchai@redhat.com> * hash_table_bench.cc: fix build without gflags Signed-off-by: NKefu Chai <kchai@redhat.com> * remove gtest from librocksdb linkage testharness.cc is included in librocksdb sources, and it uses gtest. but gtest is not supposed to be part of the public API of librocksdb. so, in this change, the testharness.cc is moved out out librocksdb, and is built as an object target, then linked with the tools and tests instead. Signed-off-by: NMarcus Watts <mwatts@redhat.com> Signed-off-by: NKefu Chai <kchai@redhat.com>
-
- 24 9月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Use ImmutableDBOptions/MutableDBOptions internally and DBOptions only for user-facing APIs. MutableDBOptions is barely a placeholder for now. I'll start to move options to MutableDBOptions in following diffs. Test Plan: make all check Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64065
-
- 08 9月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Simply move the tests Test Plan: make check -j64 Reviewers: andrewkr, lightmark, yiwu, yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62529
-
- 06 9月, 2016 1 次提交
-
-
由 Injun Song 提交于
Add 'cf_options' to source list and db_imple.cc fix casting
-
- 27 8月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: This diff split ThreadPool to -ThreadPool (abstract interface exposed in include/rocksdb/threadpool.h) -ThreadPoolImpl (actual implementation in util/threadpool_imp.h) This allow us to expose ThreadPool to the user so we can use it as an option later Test Plan: existing unit tests Reviewers: andrewkr, yiwu, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62085
-
- 20 8月, 2016 2 次提交
-
-
由 Yi Wu 提交于
Summary: Add mid-point insertion functionality to LRU cache. Caller of `Cache::Insert()` can set an additional parameter to make a cache entry have higher priority. The LRU cache will reserve at most `capacity * high_pri_pool_pct` bytes for high-pri cache entries. If `high_pri_pool_pct` is zero, the cache degenerates to normal LRU cache. Context: If we are to put index and filter blocks into RocksDB block cache, index/filter block can be swap out too early. We want to add an option to RocksDB to reserve some capacity in block cache just for index/filter blocks, to mitigate the issue. In later diffs I'll update block based table reader to use the interface to cache index/filter blocks at high priority, and expose the option to `DBOptions` and make it dynamic changeable. Test Plan: unit test. Reviewers: IslamAbdelRahman, sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D61977
-
由 Yi Wu 提交于
Summary: Clock-based cache implemenetation aim to have better concurreny than default LRU cache. See inline comments for implementation details. Test Plan: Update cache_test to run on both LRUCache and ClockCache. Adding some new tests to catch some of the bugs that I fixed while implementing the cache. Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61647
-
- 11 8月, 2016 1 次提交
-
-
由 sdong 提交于
Summary: This is a proof of concept of a RocksDB blob log file. The actual value of the Put() is appended to a blob log using normal data block format, and the handle of the block is written as the value of the key in RocksDB. The prototype only supports Put() and Get(). It doesn't support DB restart, garbage collection, Write() call, iterator, snapshots, etc. Test Plan: Add unit tests. Reviewers: arahut Reviewed By: arahut Subscribers: kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61485
-
- 06 8月, 2016 2 次提交
-
-
由 omegaga 提交于
Summary: Implement a time series database that supports DateTieredCompactionStrategy. It wraps a db object and separate SST files in different column families (time windows). Test Plan: Add `date_tiered_test`. Reviewers: dhruba, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61653
-
由 sdong 提交于
Summary: Add a utility function that trigger necessary full compaction and put output to the correct level by looking at new options and old options. Test Plan: Add unit tests for it. Reviewers: andrewkr, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: muthu, sumeet, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60783
-
- 03 8月, 2016 1 次提交
-
-
由 Dmitri Smirnov 提交于
-