- 01 8月, 2014 5 次提交
-
-
由 Igor Canadi 提交于
Summary: I was bit by this when developing SpatialDB. In case all files are at level 0, CompactRange() will output the compacted files to level 0. This is not ideal, since read amp. is much better at level 1 and higher. Test Plan: Compacted data in SpatialDB, read manifest using ldb, verified that files are now at level 1 instead of 0. Reviewers: sdong, ljin, yhchiang, dhruba Reviewed By: dhruba Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20901
-
由 Radheshyam Balasundaram 提交于
Summary: Made some small changes to fix the broken mac build Test Plan: make check all in both linux and mac. All tests pass. Reviewers: sdong, igor, ljin, yhchiang Reviewed By: ljin, yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20895
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compilation error detected in mac: db/db_test.cc:2524:3: note: in instantiation of function template specialization 'rocksdb::test::Tester::IsEq<unsigned long long, int>' requested here ASSERT_EQ(int_num, 0); ^ Test Plan: make
-
由 Yueh-Hsuan Chiang 提交于
Summary: Previously, builder.cc has a check for merge operator which prevents RocksDB from crash when reopening a DB w/o properly specifying the merge operator. However, currently we observed a memory leak on failing in RocksDB recovery. This diff removes such check and let it crash instead of causing memory leak for now before we have identified the real cause of the memory leak. Test Plan: make all check Reviewers: sdong Subscribers: ljin, igor Differential Revision: https://reviews.facebook.net/D20913
-
由 Yueh-Hsuan Chiang 提交于
-
- 31 7月, 2014 10 次提交
-
-
由 Feng Zhu 提交于
Summary: When compression is enabled, and blocksize is not too big, use the space in stack to hold bytes read from block. Bencmark: base version: commit 8f09d53f malloc: 1.30% -> 0.98% free: 1.49% -> 1.07% Test Plan: make all check Reviewers: ljin, yhchiang, dhruba, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20679
-
由 Yueh-Hsuan Chiang 提交于
fix project name in the comments
-
由 Demon 提交于
fix project name in the comments
-
由 Stanislau Hlebik 提交于
Summary: DBImplReadOnly::CompactRange wasn't override DBImpl::CompactRange; this can cause problem when using StackableDB inheritors like DbWithTtl. P. S. Thanks C++11 for override :) Test Plan: make all check Reviewers: igor, sdong Reviewed By: sdong Subscribers: yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D20829
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a typo in the comment for merge operator. Test Plan: n/a
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the crash when merge_operator is not properly set after reopen and added two test cases for this. Test Plan: make merge_test ./merge_test Reviewers: igor, ljin, sdong Reviewed By: sdong Subscribers: benj, mvikjord, leveldb Differential Revision: https://reviews.facebook.net/D20793
-
由 Feng Zhu 提交于
Summary: Define Block::Iter to be an independent class to be used by block_based_table_reader When creating data and index iterator, update an existing iterator rather than new one Thus malloc and free could be reduced Benchmark, Base: commit 76286ee6 commands: --db=/dev/shm/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --write_buffer_size=134217728 --max_write_buffer_number=2 --target_file_size_base=33554432 --max_bytes_for_level_base=1073741824 --verify_checksum=false --max_background_compactions=4 --use_plain_table=0 --memtablerep=prefix_hash --open_files=-1 --mmap_read=1 --mmap_write=0 --bloom_bits=10 --bloom_locality=1 --memtable_bloom_bits=500000 --compression_type=lz4 --num=2621440 --use_hash_search=1 --block_size=1024 --block_restart_interval=1 --use_existing_db=1 --threads=1 --benchmarks=readrandom —disable_auto_compactions=1 malloc: 3.30% -> 1.42% free: 3.59%->1.61% Test Plan: make all check run db_stress valgrind ./db_test ./table_test Reviewers: ljin, yhchiang, dhruba, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20655
-
由 Stanislau Hlebik 提交于
Summary: const string& dbname parameter is not used Test Plan: make all Reviewers: sdong, igor Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20703
-
由 sdong 提交于
Summary: We now reads table properties in VersionSet::LogAndApply(), which requires options.db_paths to be set. But since ldb_cmd directly creates VersionSet without initialization db_paths, causing a seg fault. This patch fix it by initializing db_paths. log_and_apply_bench still shows segfault, because table cache is nullptr in VersionSet created. Test Plan: Run ldb dump_manifest which used to fail. Reviewers: yhchiang, ljin, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20751
-
由 Igor Canadi 提交于
Summary: See https://github.com/facebook/rocksdb/issues/215 Test Plan: none Reviewers: dhruba, sdong, ljin, yhchiang, nkg- Reviewed By: nkg- Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20769
-
- 29 7月, 2014 7 次提交
-
-
由 Radheshyam Balasundaram 提交于
Summary: - Copy the key and value to in-memory hash table during Add operation. Also modified cuckoo_table_reader_test to use this. - Store only the user_key in in-memory hash table if it is last level file. - Handle Carryover while chosing unused key in Finish() method in case unused key was never found before Finish() call. Test Plan: cuckoo_table_reader_test --enable_perf cuckoo_table_builder_test valgrind_check asan_check Reviewers: sdong, yhchiang, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20715
-
由 sdong 提交于
Summary: We have quite some properties that are integers and we are adding more. Add a function to directly return them as an integer, instead of a string Test Plan: Add several unit test checks Reviewers: yhchiang, igor, dhruba, haobo, ljin Reviewed By: ljin Subscribers: yoshinorim, leveldb Differential Revision: https://reviews.facebook.net/D20637
-
由 sdong 提交于
Summary: Add a DB property of estimated number of live keys, by adding number of entries of all mem tables and all files, subtracted by all deletions in all files. Test Plan: Add the case in unit tests Reviewers: hobbymanyp, ljin Reviewed By: ljin Subscribers: MarkCallaghan, yoshinorim, leveldb, igor, dhruba Differential Revision: https://reviews.facebook.net/D20631
-
由 Lei Jin 提交于
Summary: It has one-to-one relationship with CFD. Take a pointer to CFD on constructor to avoid passing cfd through member functions. Test Plan: make Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20565
-
由 Lei Jin 提交于
Summary: So that we can avoid calling NowSecs() in MakeRoomForWrite twice Test Plan: make all check Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20529
-
由 Lei Jin 提交于
Summary: Make StatisticsImpl being able to forward stats to provided statistics implementation. The main purpose is to allow us to collect internal stats in the future even when user supplies custom statistics implementation. It avoids intrumenting 2 sets of stats collection code. One immediate use case is tuning advisor, which needs to collect some internal stats, users may not be interested. Test Plan: ran db_bench and see stats show up at the end of run Will run make all check since some tests rely on statistics Reviewers: yhchiang, sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20145
-
由 Lei Jin 提交于
Summary: Make StatisticsImpl being able to forward stats to provided statistics implementation. The main purpose is to allow us to collect internal stats in the future even when user supplies custom statistics implementation. It avoids intrumenting 2 sets of stats collection code. One immediate use case is tuning advisor, which needs to collect some internal stats, users may not be interested. Test Plan: ran db_bench and see stats show up at the end of run Will run make all check since some tests rely on statistics Reviewers: yhchiang, sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20145
-
- 26 7月, 2014 4 次提交
-
-
由 sdong 提交于
Summary: In block based table's hash index checking, when looking for a key that doesn't exist, there is a high chance that a false block is returned because of hash bucket conflicts. In this revision, another check is done to filter out some of those cases: comparing previous key of the block boundary to see whether the target block is what we are looking for. In a favored test setting (bloom filter disabled, 8 L0 files), I saw about 80% improvements. In a non-favored test setting (bloom filter enabled, files are all in L1, files are all cached), I see the performance penalty is less than 3%. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: ljin Subscribers: wuj, leveldb, zagfox, yhchiang Differential Revision: https://reviews.facebook.net/D20595
-
由 Radheshyam Balasundaram 提交于
Summary: Contains: - Implementation of TableReader based on Cuckoo Hashing - Unittests for CuckooTableReader - Performance test for TableReader Test Plan: make cuckoo_table_reader_test ./cuckoo_table_reader_test make valgrind_check make asan_check Reviewers: yhchiang, sdong, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20511
-
由 Lei Jin 提交于
Summary: User gets undefinied error since the definition is not exposed. Also re-enable the db test with only upper bound check Test Plan: db_test, rate_limit_test Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20403
-
由 Igor Canadi 提交于
-
- 25 7月, 2014 5 次提交
-
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed compaction-related errors where number of input levels are hard-coded. It's a bug found in compaction branch. This diff will be pushed into master. Test Plan: export ROCKSDB_TESTS=Compact make db_test -j32 ./db_test also passed the tests in compaction branch Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20577
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: I changed SpatialDB API so that we only specify list of indexes when we create the database. That way, whoever is querying the DB doesn't need to know the full list of indexes and their options. Test Plan: spatial_db_test Reviewers: yinwang Reviewed By: yinwang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20571
-
由 Radheshyam Balasundaram 提交于
Summary: Contains the following changes in CuckooTableBuilder: - Take an extra parameter in constructor to identify last level file. - Implement a better way to identify if a bucket has been inserted into the tree already during BFS search. - Minor typos Test Plan: make cuckoo_table_builder ./cuckoo_table_builder make valgrind_check Reviewers: sdong, igor, yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20445
-
- 24 7月, 2014 9 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add compression type to options make rocksdbjava make sample Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike Reviewed By: yhchiang, sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D20463
-
由 Igor Canadi 提交于
Summary: As title. Test Plan: Added test to column_family_test Reviewers: ljin, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20523
-
由 Igor Canadi 提交于
-
由 Feng Zhu 提交于
Summary: Modify a functioin TrimAppend in dbformat.h: IterKey. Write a test for it in dbformat_test Use IterKey in block::Iter to replace std::string to reduce malloc. Evaluate it using perf record. malloc: 4.26% -> 2.91% free: 3.61% -> 3.08% Test Plan: make all check ./valgrind db_test dbformat_test Reviewers: ljin, haobo, yhchiang, dhruba, igor, sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D20433
-
由 Igor Canadi 提交于
Summary: This diff is adding spatial index support to RocksDB. When creating the DB user specifies a list of spatial indexes. Spatial indexes can cover different areas and have different resolution (i.e. number of tiles). This is useful for supporting different zoom levels. Each element inserted into SpatialDB has: * a bounding box, which determines how will the element be indexed * string blob, which will usually be WKB representation of the polygon (http://en.wikipedia.org/wiki/Well-known_text) * feature set, which is a map of key-value pairs, where value can be int, double, bool, null or a string. FeatureSet will be a set of tags associated with geo elements (for example, 'road': 'highway' and similar) * a list of indexes to insert the element in. For example, small river element will be inserted in index for high zoom level, while country border will be inserted in all indexes (including the index for low zoom level). Each query is executed on single spatial index. Query guarantees that it will return all elements intersecting the specified bounding box, but it might also return some extra non-intersecting elements. Test Plan: Added bunch of unit tests in spatial_db_test Reviewers: dhruba, yinwang Reviewed By: yinwang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20361
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add the missing ROCKSDB_JAR variable in Makefile, which is mistakenly removed in https://reviews.facebook.net/D20289. Test Plan: export ROCKSDB_JAR= make rocksdbjava
-
由 Yueh-Hsuan Chiang 提交于
Summary: Update header inclusion as include/utilities/*.h has been moved to include/rocksdb/utilities/*.h Test Plan: make rocksdbjava Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20517
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compile error by replacing pow by shift, as it computes power of 2. util/options_builder.cc:133:14: error: no member named 'pow' in namespace 'std' std::pow(2, std::max(0, std::min(3, level0_stop_writes_trigger - ~~~~~^ 1 error generated. make: *** [util/options_builder.o] Error 1 Test Plan: make success in mac and linux Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20475
-