- 29 3月, 2014 5 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev:
-
由 Dhruba Borthakur 提交于
Summary: This patch stores gps locations in rocksdb. Each object is uniquely identified by an id. Each object has a gps (latitude, longitude) associated with it. The geodb supports looking up an object either by its gps location or by its id. There is a method to retrieve all objects within a circular radius centered at a specified gps location. Test Plan: Simple unit-test attached. Reviewers: leveldb, haobo Reviewed By: haobo CC: leveldb, tecbot, haobo Differential Revision: https://reviews.facebook.net/D15567
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
-
由 Lei Jin 提交于
Summary: By constraining the probes within cache line(s), we can improve the cache miss rate thus performance. This probably only makes sense for in-memory workload so defaults the option to off. Numbers and comparision can be found in wiki: https://our.intern.facebook.com/intern/wiki/index.php/Ljin/rocksdb_perf/2014_03_17#Bloom_Filter_Study Test Plan: benchmarked this change substantially. Will run make all check as well Reviewers: haobo, igor, dhruba, sdong, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17133
-
- 28 3月, 2014 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix the bug in MergeUtil which causes mixing values of different keys. Test Plan: stringappend_test make all check Reviewers: haobo, igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17235
-
由 Haobo Xu 提交于
Summary: as title Test Plan: db_test Reviewers: sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D17217
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
- 27 3月, 2014 6 次提交
-
-
由 Igor Canadi 提交于
Summary: NewFixedPrefixTransform is leaked in default options. Broken by https://github.com/facebook/rocksdb/commit/b47812fba601e23872349407d565d15f0b41a2fe Also included in the diff some code cleanup Test Plan: valgrind env_test also make check Reviewers: haobo, danguo, yhchiang Reviewed By: danguo CC: leveldb Differential Revision: https://reviews.facebook.net/D17211
-
由 sdong 提交于
Summary: Compiler complains some errors when building using our internal build settings. Fix them. Test Plan: rebuild Reviewers: haobo, dhruba, igor, yhchiang, ljin Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17199
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Previously, we used to sort all files by BySmallestFirst comparator and then re-sort level0 files in the Finalize() (recently moved to end of SaveTo). In this diff, I chose the correct comparator at the beginning and sort the files correctly in Builder::SaveTo. I also added a verification that all files are sorted correctly in CheckConsistency() NOTE: This diff depends on D17037 Test Plan: make check. Will also run db_stress Reviewers: dhruba, haobo, sdong, ljin Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17049
-
由 Igor Canadi 提交于
Summary: AssertHeld() was a no-op before. Now it does things. Also, this change caught a bad bug in SuperVersion::Init(). The method is calling db->mutex.AssertHeld(), but db variable is not initialized yet! I also fixed that issue. Test Plan: make check Reviewers: dhruba, haobo, ljin, sdong, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17193
-
由 Igor Canadi 提交于
Summary: We don't preallocate MANIFEST file, even though we have an option for that. This diff preallocates manifest file every time we create it Test Plan: make check Reviewers: dhruba, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17163
-
- 26 3月, 2014 6 次提交
-
-
由 sdong 提交于
Summary: This is the part that was not finished when doing the Options.max_num_files=-1 feature. For iterating non level0 SST files (which was done using two level iterator), table cache is not bypassed. With this patch, the leftover feature is done. Test Plan: make all check; change Options.max_num_files=-1 in one of the tests to cover the codes. Reviewers: haobo, igor, dhruba, ljin, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17001
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add assert(operands_list.size() >= 2) in MergeOperator::PartialMergeMulti to ensure it's only be called when we have at least two merge operands. Test Plan: run merge_test and stringappend_test. Reviewers: haobo, igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17169
-
由 Igor Canadi 提交于
-
由 Danny Guo 提交于
Summary: Currently if client uses kNULLString as the prefix, it will confuse compaction filter v2. This diff added a bool to indicate if the prefix has been intialized. I also added a unit test to cover this case and make sure the new code path is hit. Test Plan: db_test Reviewers: igor, haobo Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17151
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix a bug that PartialMergeMulti will try to merge the first operand with an empty slice. Test Plan: run stringappend_test and merge_test. Reviewers: haobo, igor Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17157
-
由 Igor Canadi 提交于
-
- 25 3月, 2014 4 次提交
-
-
由 Danny Guo 提交于
Summary: This diff adds a new CompactionFilterV2 API that roll up the decisions of kv pairs during compactions. These kv pairs must share the same key prefix. They are buffered inside the db. typedef std::vector<Slice> SliceVector; virtual std::vector<bool> Filter(int level, const SliceVector& keys, const SliceVector& existing_values, std::vector<std::string>* new_values, std::vector<bool>* values_changed ) const = 0; Application can override the Filter() function to operate on the buffered kv pairs. More details in the inline documentation. Test Plan: make check. Added unit tests to make sure Keep, Delete, Change all works. Reviewers: haobo CCs: leveldb Differential Revision: https://reviews.facebook.net/D15087
-
由 Yueh-Hsuan Chiang 提交于
Summary: * PartialMerge api now takes a list of operands instead of two operands. * Add min_pertial_merge_operands to Options, indicating the minimum number of operands to trigger partial merge. * This diff is based on Schalk's previous diff (D14601), but it also includes necessary changes such as updating the pure C api for partial merge. Test Plan: * make check all * develop tests for cases where partial merge takes more than two operands. TODOs (from Schalk): * Add test with min_partial_merge_operands > 2. * Perform benchmarks to measure the performance improvements (can probably use results of task #2837810.) * Add description of problem to doc/index.html. * Change wiki pages to reflect the interface changes. Reviewers: haobo, igor, vamsi Reviewed By: haobo CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D16815
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Might be useful if client doesn't want to effect running system during backup too much. Test Plan: added a test case Reviewers: dhruba, haobo, ljin Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17091
-
- 22 3月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: LogBuffer::AddLogToBuffer() uses vsnprintf() in the wrong way, which might cause buffer overflow when log line is too line. Fix it. Test Plan: Add a unit test to cover most LogBuffer's most logic. Reviewers: igor, haobo, dhruba Reviewed By: igor CC: ljin, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D17103
-
- 21 3月, 2014 6 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Everytime a client opens a DB, we do a sanity check that: * checks the existance of all the necessary files * verifies that file sizes are correct Some of the code was stolen from https://reviews.facebook.net/D16935 Test Plan: added a unit test Reviewers: dhruba, haobo, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D17097
-
由 Yiting Li 提交于
Summary: Added a function/command to check the consistency of live files' meta data Test Plan: Manual test (size mismatch, file not exist). Command test script. Reviewers: haobo Reviewed By: haobo CC: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D16935
-
由 Igor Canadi 提交于
Summary: Whenever we get an IOError from GetImpl() or NewIterator(), we should immediatelly mark the DB read-only. The same check already exists in Write() and Compaction(). This should help with clients that are somehow missing a file. Test Plan: make check Reviewers: dhruba, haobo, sdong, ljin Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D17061
-
由 Igor Canadi 提交于
Summary: We should clean up after ourselves. Last crashtest failed because the disk on the cibuild machine was full. Also, db_crashtest is supposed to run the test on the same database in every iteration. This isn't the case currently, because every run creates a new tmp directory. It's fixed with this diff. Test Plan: ran it Reviewers: ljin Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17073
-
由 sdong 提交于
Summary: D17067 breaks DBTest.UniversalCompactionTrigger because of wrong location of the checking. Fix it. Test Plan: Run the test and make sure it passes. Reviewers: igor, haobo Reviewed By: igor CC: dhruba, ljin, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D17079
-
- 20 3月, 2014 7 次提交
-
-
由 sdong 提交于
Summary: Add unit tests to make sure CompactionFilterContext::is_manual_compaction_ and CompactionFilterContext::is_full_compaction_ are set correctly. Test Plan: run the new tests. Reviewers: haobo, igor, dhruba, yhchiang, ljin Reviewed By: haobo CC: nkg-, leveldb Differential Revision: https://reviews.facebook.net/D17067
-
由 Igor Canadi 提交于
Summary: As it turns out, we need the call to ComputeCompactionScore (previously: Finalize) in CompactionPicker. The issue caused a deadlock in db_stress: http://ci-builds.fb.com/job/rocksdb_crashtest/290/console The last two lines before a deadlock were: 2014/03/18-22:43:41.481029 7facafbee700 (Original Log Time 2014/03/18-22:43:41.480989) Compaction nothing to do 2014/03/18-22:43:41.481041 7faccf7fc700 wait for fewer level0 files... "Compaction nothing to do" and other thread waiting for fewer level0 files. Hm hm. I moved the pre-sorting to SaveTo, which should fix both the original and the new issue. Test Plan: make check for now, will run db_stress in jenkins Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17037
-
由 Kai Liu 提交于
Summary: Previous code had two bugs: * didn't initialize the table_magic_number_ explicitly -- as a result a random junk number is stored for table_magic_number_, making HasInitializedMagicNumber() always return true. * if condition is inconrrect in set_table_magic_number(), and the return value is not checked. I replace if-else by a stronger requirement enforced by assert(). Test Plan: Previous sst_dump failed to work. After the fix, things back to normal. Reviewers: yhchiang CC: haobo, sdong, igor, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17055
-
由 Igor Canadi 提交于
Summary: We have an issue with internal service trying to run compaction with zero input files: 2014/02/07-02:26:58.386531 7f79117ec700 Compaction start summary: Base version 1420 Base level 3, seek compaction:0, inputs:[ϛ~^Qy^?],[] 2014/02/07-02:26:58.386539 7f79117ec700 Compacted 0@3 + 0@4 files => 0 bytes There are two issues: * inputsummary is printing out junk * it's constantly retrying (since I guess madeProgress is true), so it prints out a lot of data in the LOG file (40GB in one day). I read through the Level compaction picker and added some failure condition if input[0] is empty. I think PickCompaction() should not return compaction with zero input files with this change. I'm not confident enough to add an assertion though :) Test Plan: make check Reviewers: dhruba, haobo, sdong, kailiu Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16005
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Compile issues are: * Unused variable env_ * Unused fallocate_with_keep_size_ Test Plan: compiles Reviewers: dhruba, haobo, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D17043
-
由 Lei Jin 提交于
Summary: This is a 500ns operation while the whole Get() call takes only a few micro! Test Plan: ran db_bench, for a DB with 50M keys, QPS jumps from 5.2M/s to 7.2M/s Reviewers: haobo, igor, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17007
-
- 19 3月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: Add a property to calculate number of background errors encountered to help users build their monitoring Test Plan: Add a unit test. make all check Reviewers: haobo, igor, dhruba Reviewed By: igor CC: ljin, nkg-, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D16959
-