- 21 1月, 2017 1 次提交
-
-
由 Kefu Chai 提交于
Summary: because not all archs support this option. see https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html. also do not pass "-fno-omit-frame-pointer" and "-momit-leaf-frame-pointer" to compiler if ${CMAKE_BUILD_TYPE} is "Debug". this matches the behaviour of DEBUG_LEVEL=2 in Makefile. Signed-off-by: NKefu Chai <tchaikov@gmail.com> Closes https://github.com/facebook/rocksdb/pull/1762 Differential Revision: D4444036 Pulled By: yiwu-arbug fbshipit-source-id: 8596fbe
-
- 17 12月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Iterator should be in corrupted status if merge operator return false. Also add test to make sure if max_successive_merges is hit during write, data will not be lost. Closes https://github.com/facebook/rocksdb/pull/1665 Differential Revision: D4322695 Pulled By: yiwu-arbug fbshipit-source-id: b327b05
-
- 02 12月, 2016 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Now that we have userspace persisted cache, we don't need flashcache anymore. Closes https://github.com/facebook/rocksdb/pull/1588 Differential Revision: D4245114 Pulled By: igorcanadi fbshipit-source-id: e2c1c72
-
- 30 11月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/1578 Differential Revision: D4241171 Pulled By: ajkr fbshipit-source-id: ce5fd83
-
- 19 11月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
Summary: Signed-off-by: NKefu Chai <tchaikov@gmail.com> Closes https://github.com/facebook/rocksdb/pull/1540 Differential Revision: D4207564 Pulled By: siying fbshipit-source-id: 567415b
-
- 17 11月, 2016 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff includes an implementation of CompactionFilter that allows users to write CompactionFilter in Lua. With this ability, users can dynamically change compaction filter logic without requiring building the rocksdb binary and restarting the database. To compile, WITH_LUA_PATH must be specified to the base directory of lua. Closes https://github.com/facebook/rocksdb/pull/1478 Differential Revision: D4150138 Pulled By: yhchiang fbshipit-source-id: ed84222
-
- 16 11月, 2016 1 次提交
-
-
由 Willem Jan Withagen 提交于
Summary: This will allow reference to `malloc_stats_print` Closes https://github.com/facebook/rocksdb/pull/1516 Differential Revision: D4187258 Pulled By: siying fbshipit-source-id: 34ae9f9
-
- 14 11月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Currently our skip-list have an optimization to speedup sequential inserts from a single stream, by remembering the last insert position. We extend the idea to support sequential inserts from multiple streams, and even tolerate small reordering wihtin each stream. This PR is the interface part adding the following: - Add `memtable_insert_prefix_extractor` to allow specifying prefix for each key. - Add `InsertWithHint()` interface to memtable, to allow underlying implementation to return a hint of insert position, which can be later pass back to optimize inserts. - Memtable will maintain a map from prefix to hints and pass the hint via `InsertWithHint()` if `memtable_insert_prefix_extractor` is non-null. Closes https://github.com/facebook/rocksdb/pull/1419 Differential Revision: D4079367 Pulled By: yiwu-arbug fbshipit-source-id: 3555326
-
- 01 11月, 2016 1 次提交
-
-
由 Vladislav Vaintroub 提交于
Summary: If FAIL_ON_WARNINGS is OFF, compilation will succeed if there were compile warnings. Closes https://github.com/facebook/rocksdb/pull/1446 Differential Revision: D4107825 Pulled By: yiwu-arbug fbshipit-source-id: 4ce6e05
-
- 29 10月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
this breaks the cross-compiling, and we can not assume that the building machine and the target machine share the same CPU spec. Signed-off-by: NKefu Chai <kchai@redhat.com>
-
- 26 10月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
* util/build_verion.cc.in: add this file, so cmake and make can share the template file for generating util/build_version.cc. * CMakeLists.txt: also, cmake v2.8.11 does not support file(GENERATE ...), so we are using configure_file() for creating build_version.cc. * Makefile: use util/build_verion.cc.in for creating build_version.cc. Signed-off-by: NKefu Chai <tchaikov@gmail.com>
-
- 21 10月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Changes in the diff API changes: - Introduce IngestExternalFile to replace AddFile (I think this make the API more clear) - Introduce IngestExternalFileOptions (This struct will encapsulate the options for ingesting the external file) - Deprecate AddFile() API Logic changes: - If our file overlap with the memtable we will flush the memtable - We will find the first level in the LSM tree that our file key range overlap with the keys in it - We will find the lowest level in the LSM tree above the the level we found in step 2 that our file can fit in and ingest our file in it - We will assign a global sequence number to our new file - Remove AddFile restrictions by using global sequence numbers Other changes: - Refactor all AddFile logic to be encapsulated in ExternalSstFileIngestionJob Test Plan: unit tests (still need to add more) addfile_stress (https://reviews.facebook.net/D65037) Reviewers: yiwu, andrewkr, lightmark, yhchiang, sdong Reviewed By: sdong Subscribers: jkedgar, hcz, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D65061
-
- 19 10月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This diff introduces RangeDelAggregator, which takes ownership of iterators provided to it via AddTombstones(). The tombstones are organized in a two-level map (snapshot stripe -> begin key -> tombstone). Tombstone creation avoids data copy by holding Slices returned by the iterator, which remain valid thanks to pinning. For compaction, we create a hierarchical range tombstone iterator with structure matching the iterator over compaction input data. An aggregator based on that iterator is used by CompactionIterator to determine which keys are covered by range tombstones. In case of merge operand, the same aggregator is used by MergeHelper. Upon finishing each file in the compaction, relevant range tombstones are added to the output file's range tombstone metablock and file boundaries are updated accordingly. To check whether a key is covered by range tombstone, RangeDelAggregator::ShouldDelete() considers tombstones in the key's snapshot stripe. When this function is used outside of compaction, it also checks newer stripes, which can contain covering tombstones. Currently the intra-stripe check involves a linear scan; however, in the future we plan to collapse ranges within a stripe such that binary search can be used. RangeDelAggregator::AddToBuilder() adds all range tombstones in the table's key-range to a new table's range tombstone meta-block. Since range tombstones may fall in the gap between files, we may need to extend some files' key-ranges. The strategy is (1) first file extends as far left as possible and other files do not extend left, (2) all files extend right until either the start of the next file or the end of the last range tombstone in the gap, whichever comes first. One other notable change is adding release/move semantics to ScopedArenaIterator such that it can be used to transfer ownership of an arena-allocated iterator, similar to how unique_ptr is used for malloc'd data. Depends on D61473 Test Plan: compaction_iterator_test, mock_table, end-to-end tests in D63927 Reviewers: sdong, IslamAbdelRahman, wanning, yhchiang, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62205
-
- 13 10月, 2016 1 次提交
-
-
- 11 10月, 2016 1 次提交
-
-
由 Bassam Tabbara 提交于
Signed-off-by: NBassam Tabbara <bassam.tabbara@quantum.com>
-
- 29 9月, 2016 1 次提交
-
-
由 Kefu Chai 提交于
* enable cmake to work on linux and osx also * port part of build_detect_platform not covered by thirdparty.inc to cmake. - detect fallocate() - detect malloc_usable_size() - detect JeMalloc - detect snappy * check for asan,tsan,ubsan * create 'build_version.cc' in build directory. * add `check` target to support 'make check'. * add `tools` target to match its counterpart in Makefile. * use `date` on non-win32 platforms. * pass different cflags on non-win32 platforms * detect pthead library using FindThread cmake module. * enable CMP0042 to silence the cmake warning on osx * reorder the linked libraries. because testutillib references gtest, to enable the linker to find the referenced symbols, we need to put gtest after testutillib. Signed-off-by: NMarcus Watts <mwatts@redhat.com> Signed-off-by: NKefu Chai <kchai@redhat.com> * hash_table_bench.cc: fix build without gflags Signed-off-by: NKefu Chai <kchai@redhat.com> * remove gtest from librocksdb linkage testharness.cc is included in librocksdb sources, and it uses gtest. but gtest is not supposed to be part of the public API of librocksdb. so, in this change, the testharness.cc is moved out out librocksdb, and is built as an object target, then linked with the tools and tests instead. Signed-off-by: NMarcus Watts <mwatts@redhat.com> Signed-off-by: NKefu Chai <kchai@redhat.com>
-
- 24 9月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Use ImmutableDBOptions/MutableDBOptions internally and DBOptions only for user-facing APIs. MutableDBOptions is barely a placeholder for now. I'll start to move options to MutableDBOptions in following diffs. Test Plan: make all check Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64065
-
- 08 9月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Simply move the tests Test Plan: make check -j64 Reviewers: andrewkr, lightmark, yiwu, yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62529
-
- 06 9月, 2016 1 次提交
-
-
由 Injun Song 提交于
Add 'cf_options' to source list and db_imple.cc fix casting
-
- 27 8月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: This diff split ThreadPool to -ThreadPool (abstract interface exposed in include/rocksdb/threadpool.h) -ThreadPoolImpl (actual implementation in util/threadpool_imp.h) This allow us to expose ThreadPool to the user so we can use it as an option later Test Plan: existing unit tests Reviewers: andrewkr, yiwu, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62085
-
- 20 8月, 2016 2 次提交
-
-
由 Yi Wu 提交于
Summary: Add mid-point insertion functionality to LRU cache. Caller of `Cache::Insert()` can set an additional parameter to make a cache entry have higher priority. The LRU cache will reserve at most `capacity * high_pri_pool_pct` bytes for high-pri cache entries. If `high_pri_pool_pct` is zero, the cache degenerates to normal LRU cache. Context: If we are to put index and filter blocks into RocksDB block cache, index/filter block can be swap out too early. We want to add an option to RocksDB to reserve some capacity in block cache just for index/filter blocks, to mitigate the issue. In later diffs I'll update block based table reader to use the interface to cache index/filter blocks at high priority, and expose the option to `DBOptions` and make it dynamic changeable. Test Plan: unit test. Reviewers: IslamAbdelRahman, sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D61977
-
由 Yi Wu 提交于
Summary: Clock-based cache implemenetation aim to have better concurreny than default LRU cache. See inline comments for implementation details. Test Plan: Update cache_test to run on both LRUCache and ClockCache. Adding some new tests to catch some of the bugs that I fixed while implementing the cache. Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61647
-
- 11 8月, 2016 1 次提交
-
-
由 sdong 提交于
Summary: This is a proof of concept of a RocksDB blob log file. The actual value of the Put() is appended to a blob log using normal data block format, and the handle of the block is written as the value of the key in RocksDB. The prototype only supports Put() and Get(). It doesn't support DB restart, garbage collection, Write() call, iterator, snapshots, etc. Test Plan: Add unit tests. Reviewers: arahut Reviewed By: arahut Subscribers: kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61485
-
- 06 8月, 2016 2 次提交
-
-
由 omegaga 提交于
Summary: Implement a time series database that supports DateTieredCompactionStrategy. It wraps a db object and separate SST files in different column families (time windows). Test Plan: Add `date_tiered_test`. Reviewers: dhruba, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61653
-
由 sdong 提交于
Summary: Add a utility function that trigger necessary full compaction and put output to the correct level by looking at new options and old options. Test Plan: Add unit tests for it. Reviewers: andrewkr, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: muthu, sumeet, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60783
-
- 03 8月, 2016 1 次提交
-
-
由 Dmitri Smirnov 提交于
-
- 02 8月, 2016 1 次提交
-
-
由 omegaga 提交于
Summary: Experiments on column-aware encodings. Supported features: 1) extract data blocks from SST file and encode with specified encodings; 2) Decode encoded data back into row format; 3) Directly extract data blocks and write in row format (without prefix encoding); 4) Get column distribution statistics for column format; 5) Dump data blocks separated by columns in human-readable format. There is still on-going work on this diff. More refactoring is necessary. Test Plan: Wrote tests in `column_aware_encoding_test.cc`. More tests should be added. Reviewers: sdong Reviewed By: sdong Subscribers: arahut, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60027
-
- 27 7月, 2016 1 次提交
-
-
由 Alexander Jipa 提交于
fixes 1230: Error:string sub-command REGEX, mode REPLACE needs at least 6 arguments total to command (#1231) quoting the las argument of string REGEXP REPLACE function
-
- 26 7月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: TickersNameMap is not consistent with Tickers enum. this cause us to report wrong statistics and sometimes to access TickersNameMap outside it's boundary causing crashes (in Fb303 statistics) Test Plan: added new unit test Reviewers: sdong, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61083
-
- 22 7月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Fix flush not being commit while writing manifest, which is a recent bug introduced by D60075. The issue: # Options.max_background_flushes > 1 # Background thread A pick up a flush job, flush, then commit to manifest. (Note that mutex is released before writing manifest.) # Background thread B pick up another flush job, flush. When it gets to `MemTableList::InstallMemtableFlushResults`, it notices another thread is commiting, so it quit. # After the first commit, thread A doesn't double check if there are more flush result need to commit, leaving the second flush uncommitted. Test Plan: run the test. Also verify the new test hit deadlock without the fix. Reviewers: sdong, igor, lightmark Reviewed By: lightmark Subscribers: andrewkr, omegaga, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60969
-
- 16 7月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: Refactor cache.cc so that I can plugin clock cache (D55581). Mainly move `ShardedCache` to separate file, move `LRUHandle` back to cache.cc and rename it lru_cache.cc. Test Plan: make check -j64 Reviewers: lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59655
-
- 13 7月, 2016 1 次提交
-
-
由 Yi Wu 提交于
Summary: When write stalls because of auto compaction is disabled, or stop write trigger is reached, user may change these two options to unblock writes. Unfortunately we had issue where the write thread will block the attempt to persist the options, thus creating a deadlock. This diff fix the issue and add two test cases to detect such deadlock. Test Plan: Run unit tests. Also, revert db_impl.cc to master (but don't revert `DBImpl::BackgroundCompaction:Finish` sync point) and run db_options_test. Both tests should hit deadlock. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60627
-
- 30 6月, 2016 1 次提交
-
-
由 John Alexander 提交于
* Fixed Windows build error in CMakeLists.txt and perf_level error in db_bench_tool.cc * Changed hard-coded perf levels in db_bench_tool.cc to enum values from perf_level.h * Replaced remaining FLAGS_perf_level > 0
-
- 11 6月, 2016 1 次提交
-
-
由 sdong 提交于
Summary: It confuses some compilers to have slice.cc under multiple directories. Merge them. Test Plan: Run existing tests Reviewers: andrewkr, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59409
-
- 10 6月, 2016 2 次提交
-
-
由 sdong 提交于
Summary: AFIK, options builder is not used by anyone. Remove it. Test Plan: Run all existing tests. Reviewers: IslamAbdelRahman, andrewkr, igor Reviewed By: igor Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59319
-
由 krad 提交于
Summary: Enabled build in Windows and corresponding fixes Test Plan: Compile and run persistent_cache_test in Windows and make check in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59307
-
- 07 6月, 2016 1 次提交
-
-
由 krad 提交于
Summary: This diff provides the basic interface definitions of persistent read cache system PersistentCacheOptions captures the persistent read cache options used to configure and control the system PersistentCacheTier provides the basic building block for constructing tiered cache PersistentTieredCache provides a logical abstraction of tiers of cache layered over one another Test Plan: Compile Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57051
-
- 04 6月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Extracted basic Env-related tests from mock_env_test and memenv_test into a parameterized test for Envs: env_basic_test. Depends on D58449. (The dependency is here only so I can keep this series of diffs in a chain -- there is no dependency on that diff's code.) Test Plan: ran tests Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58635
-
- 03 6月, 2016 2 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This enables configurable Envs without recompiling. For example, my next diff will make env_test test an Env created by NewEnvFromUri(). Then, users can determine which Env is tested simply by providing the URI for NewEnvFromUri() (e.g., through a CLI argument or environment variable). The registration process allows us to register any Env that is linked with the RocksDB library, so we can register our internal Envs as well. The registration code is inspired by our internal InitRegistry. Test Plan: new unit test Reviewers: IslamAbdelRahman, lightmark, ldemailly, sdong Reviewed By: sdong Subscribers: leveldb, dhruba, andrewkr Differential Revision: https://reviews.facebook.net/D58449
-
由 krad 提交于
Summary: Add hash table (under persistent cache) to CMake list Test Plan: Run hash_test in windows and make check in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59151
-