- 20 6月, 2014 9 次提交
-
-
由 Igor Canadi 提交于
Summary: After evaluating options for JSON storage, I decided to implement our own. The reason is that we'll be able to optimize it better and we get to reduce unnecessary dependencies (which is what we'd get with folly). I also plan to write a serializer/deserializer for JSONDocument with our own binary format similar to BSON. That way we'll store binary JSON format in RocksDB instead of the plain-text JSON. This means less storage and faster deserialization. There are still some inefficiencies left here. I plan to optimize them after we develop a functioning DocumentDB. That way we can move and iterate faster. Test Plan: added a unit test Reviewers: dhruba, haobo, sdong, ljin, yhchiang Reviewed By: haobo Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18831
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: As discussed in our internal group, we don't get much use of seek compaction at the moment, while it's making code more complicated and slower in some cases. This diff removes seek compaction and (hopefully) all code that was introduced to support seek compaction. There is one test case that relied on didIO information. I'll try to find another way to implement it. Test Plan: make check Reviewers: sdong, haobo, yhchiang, ljin, dhruba Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19161
-
由 Igor Canadi 提交于
Summary: We decided that one of the long term goals is to unify level and universal compaction. As a small first step, I'm unifying level 0 sorting methods. Previously, we used to sort level 0 files in level compaction by file number and in universal compaction by sequence number. But it turns out that in level compaction, sorting by file number is exactly the same as sorting by sequence number. Test Plan: Ran make check with bunch of asserts to verify the sorting order is exactly the same. Also, make check with this patch Reviewers: haobo, yhchiang, ljin, dhruba, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19131
-
由 Haobo Xu 提交于
Summary: Currently, RocksDB returns error if a db written with prefix hash index, is later opened without providing a prefix extractor. This is uncessarily harsh. Without a prefix extractor, we could always fallback to the normal binary index. Test Plan: unit test, also manually veried LOG that fallback did occur. Reviewers: sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19191
-
由 Yueh-Hsuan Chiang 提交于
Summary: Currently, when something badly happen in the DB::Write() while the write-queue contains more than one element, the current design seems to forget to clean up the queue as well as wake-up all the writers, this potentially makes rocksdb hang on writes. Test Plan: make all check Reviewers: sdong, ljin, igor, haobo Reviewed By: haobo Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19167
-
由 Igor Canadi 提交于
Support for compaction filters in the C API
-
由 Lei Jin 提交于
Summary: s/class ParsedInternalKey/struct ParsedInternalKey Test Plan: make shared_lib Reviewers: igor, yhchiang, sdong, haobo Reviewed By: haobo Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19173
-
由 Lei Jin 提交于
Summary: asan_crash_test is failing on segfault Test Plan: running asan_crash_test Reviewers: sdong, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19149
-
- 19 6月, 2014 6 次提交
-
-
由 Evan Shaw 提交于
-
由 Haobo Xu 提交于
Summary: fix a bug in D19047, which caused DBTest.RecoverDuringMemtableCompaction to fail. Test Plan: unit test Reviewers: sdong, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19155
-
由 Evan Shaw 提交于
-
由 Evan Shaw 提交于
-
由 sdong 提交于
Summary: Add a encoding feature of PlainTable to encode PlainTable's keys to save some bytes for the same prefixes. The data format is documented in table/plain_table_factory.h Test Plan: Add unit test coverage in plain_table_db_test Reviewers: yhchiang, igor, dhruba, ljin, haobo Reviewed By: haobo Subscribers: nkg-, leveldb Differential Revision: https://reviews.facebook.net/D18735
-
由 Haobo Xu 提交于
Summary: Currently, the in-memory hash index of blockbased table uses a precise hash map to track the prefix to block range mapping. In some use cases, especially when prefix itself is big, the memory overhead becomes a problem. This diff introduces a fixed hash bucket array that does not store the prefix and allows prefix collision, which is similar to the plaintable hash index, in order to reduce the memory consumption. Just a quick draft, still testing and refining. Test Plan: unit test and shadow testing Reviewers: dhruba, kailiu, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19047
-
- 18 6月, 2014 6 次提交
-
-
由 Igor Canadi 提交于
Summary: This is minor, but if we put the writing talbe factory as the third parameter, when we add a new table format, we'll have a situation: 1) block based factory 2) plain table factory 3) output factory 4) new format factory I think it makes more sense to have output as the first parameter. Also, fixed a NewAdaptiveTableFactory() call in unit test Test Plan: unit test Reviewers: sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19119
-
由 sdong 提交于
Summary: Add two parameters of hash linked list to log distribution of number of entries across all buckets, and a sample row when there are too many entries in one single bucket. Test Plan: Turn it on in plain_table_db_test and see the logs. Reviewers: haobo, ljin Reviewed By: ljin Subscribers: leveldb, nkg-, dhruba, yhchiang Differential Revision: https://reviews.facebook.net/D19095
-
由 Yueh-Hsuan Chiang 提交于
specify the command to install build_tools/mac-install-gflags.sh file in...
-
由 nawu 提交于
-
由 sdong 提交于
Summary: The new table factory is used if users want to convert a DB from one table format to the other. A user can use this table to open a DB written using one table format and write new files to another table format. Test Plan: add a unit test Reviewers: haobo, igor Reviewed By: igor Subscribers: dhruba, ljin, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D19017
-
由 Igor Canadi 提交于
Add separate Read/WriteUnlock methods in MutexRW.
-
- 17 6月, 2014 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Include max_write_buffer_number >= 2 to SanitizeOptions. Test Plan: make all check Reviewers: haobo, sdong, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19077
-
由 sdong 提交于
Summary: We added multiple fields to FileMetaData recently and are planning to add more. This refactoring separate the minimum information for accessing the file. This object is copyable (FileMetaData is not copyable since the ref counter). I hope this refactoring can enable further improvements: (1) use it to design a more efficient data structure to speed up read queries. (2) in the future, when we add information of storage level, we can easily do the encoding, instead of enlarge this structure, which might expand memory work set for file meta data. The definition is same as current EncodedFileMetaData used in two level iterator, so now the logic in two level iterator is easier to understand. Test Plan: make all check Reviewers: haobo, igor, ljin Reviewed By: ljin Subscribers: leveldb, dhruba, yhchiang Differential Revision: https://reviews.facebook.net/D18933
-
由 Bradley Grainger 提交于
Some platforms, particularly Windows, do not have a single method that can release both a held reader lock and a held writer lock; instead, a separate method (ReleaseSRWLockShared or ReleaseSRWLockExclusive) must be called in each case. This may also be necessary to back MutexRW with a shared_mutex in C++14; the current language proposal includes both an unlock() and a shared_unlock() method.
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix a bug causing LOG is not created when max_log_file_size is set. This bug is reported in issue #174. Test Plan: Add TEST(AutoRollLoggerTest, LogFileExistence). make auto_roll_logger_test ./auto_roll_logger_test Reviewers: haobo, sdong, ljin, igor, igor2 Reviewed By: igor2 Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19053
-
- 14 6月, 2014 2 次提交
-
-
由 sdong 提交于
VersionSet::Get(): Bring back the logic of skipping key range check when there are <=3 level 0 files Summary: https://reviews.facebook.net/D17205 removed the logic of skipping file key range check when there are less than 3 level 0 files. This patch brings it back. Other than that, add another small optimization to avoid to check all the levels if most higher levels don't have any file. Test Plan: make all check Reviewers: ljin Reviewed By: ljin Subscribers: yhchiang, igor, haobo, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19035
-
由 Barnaby 提交于
Moved doc/index.html to wiki
-
- 13 6月, 2014 2 次提交
-
-
由 sdong 提交于
Summary: sst_dump now doesn't work well for PlainTable. Not sure when it started, but this should fix it. Test Plan: Run sst_dump against a file that used to fail. Reviewers: yhchiang, haobo, igor Reviewed By: igor Subscribers: dhruba, ljin, leveldb Differential Revision: https://reviews.facebook.net/D19023
-
由 Lei Jin 提交于
Summary: as title Test Plan: db_bench the initial result is very promising. I will post results of complete runs Reviewers: dhruba, haobo, sdong, igor Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18867
-
- 11 6月, 2014 5 次提交
-
-
由 Lei Jin 提交于
Summary: as title Test Plan: jenkin Reviewers: igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19005
-
由 Lei Jin 提交于
Summary: as title Test Plan: push Reviewers: igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18999
-
由 sdong 提交于
Summary: In BlockBasedTable::PrefixMayMatch() we calculate prefix even if bloom is not config. Move the check before Test Plan: make all check Reviewers: igor, ljin Reviewed By: ljin Subscribers: wuj, leveldb, haobo, yhchiang, dhruba Differential Revision: https://reviews.facebook.net/D18993
-
由 Lei Jin 提交于
Summary: collect in-memory workload get/seek metrics so that we can alert on regression Test Plan: ran locally Reviewers: igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18969
-
由 Lei Jin 提交于
Summary: obvious Test Plan: db_test Reviewers: sdong, haobo, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18987
-
- 10 6月, 2014 3 次提交
-
-
由 sdong 提交于
Summary: Clean PlainTableReader's data structures: (1) inline bloom_ (in order to do this, change DynamicBloom to allow lazy initialization) (2) remove some variables only used when initialization from the class (3) put variables not used in normal read code paths to the end of the class and reference prefix_extractor directly (4) make Options a reference. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: ljin Subscribers: igor, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18891
-
由 Igor Canadi 提交于
Summary: This is a temporary solution to a issue that we have with compression libraries. See task #4453446. Test Plan: make check doesn't complain :) Reviewers: haobo, ljin, yhchiang, dhruba, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18975
-
由 sdong 提交于
Summary: Even if the file is corrupted, table properties are usually available to print out. Now sst_dump would just fail without printing table properties. With this patch, table properties are still try to be printed out. Test Plan: run sst_dump against multiple scenarios Reviewers: igor, yhchiang, ljin, haobo Reviewed By: haobo Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18981
-
- 08 6月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Improve documentation for RocksEnv and its C++ resource. Specifically, the result of RocksEnv::Default() is a singleton, and the ownership of its c++ resource belongs to rocksdb c++. As a result, calling dispose() of the return value of RocksDB::Default() will be no-op. Test Plan: no code change. Reviewers: haobo, ankgup87 Reviewed By: ankgup87 Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18945
-
- 07 6月, 2014 2 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Provide an convenience option to create column families if they are missing from the DB. Task #4460490 Test Plan: added unit test. also, stress test for some time Reviewers: sdong, haobo, dhruba, ljin, yhchiang Reviewed By: yhchiang Subscribers: yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D18951
-