- 02 6月, 2013 1 次提交
-
-
由 Haobo Xu 提交于
Summary: This diff adds an option to specify whether PTHREAD_MUTEX_ADAPTIVE_NP will be enabled for the rocksdb single big kernel lock. db_bench also have this option now. Quickly tested 8 thread cpu bound 100 byte random read. No fast mutex: ~750k/s ops With fast mutex: ~880k/s ops Test Plan: make check; db_bench; db_stress Reviewers: dhruba CC: MarkCallaghan, leveldb Differential Revision: https://reviews.facebook.net/D11031
-
- 31 5月, 2013 1 次提交
-
-
由 Haobo Xu 提交于
Summary: Current posix advice implementation ties up the access pattern hint with the creation of a file. It is not possible to apply different advice for different access (random get vs compaction read), without keeping two open files for the same table. This patch extended the RandomeAccessFile interface to accept new access hint at anytime. Particularly, we are able to set different access hint on the same table file based on when/how the file is used. Two options are added to set the access hint, after the file is first opened and after the file is being compacted. Test Plan: make check; db_stress; db_bench Reviewers: dhruba Reviewed By: dhruba CC: MarkCallaghan, leveldb Differential Revision: https://reviews.facebook.net/D10905
-
- 30 5月, 2013 1 次提交
-
-
由 Haobo Xu 提交于
Summary: As title Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10995
-
- 29 5月, 2013 2 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: Enhance the statitics to report the number of open db iterators. Test Plan: make check Reviewers: haobo, emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D10983
-
由 Haobo Xu 提交于
Summary: Overriding block_size_deviation to zero, so that CorruptionTest can pass. Test Plan: make check Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D10977
-
- 25 5月, 2013 3 次提交
-
-
由 heyongqiang 提交于
Summary: a new option block_size_deviation is added. Test Plan: run db_test and db_bench Reviewers: dhruba, haobo Reviewed By: haobo Differential Revision: https://reviews.facebook.net/D10821
-
由 heyongqiang 提交于
Summary: a new option block_size_deviation is added. Test Plan: run db_test and db_bench Reviewers: dhruba, haobo Reviewed By: haobo Differential Revision: https://reviews.facebook.net/D10821
-
由 Haobo Xu 提交于
Summary: MaybeDumpStats was causing lock problem Test Plan: make check; db_stress Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D10935
-
- 24 5月, 2013 4 次提交
-
-
由 Haobo Xu 提交于
Summary: Added an option stats_dump_period_sec to dump leveldb.stats to LOG periodically for diagnosis. By defauly, it's set to a very big number 3600 (1 hour). Test Plan: make check; Reviewers: dhruba Reviewed By: dhruba CC: leveldb, zshao Differential Revision: https://reviews.facebook.net/D10761
-
由 Dhruba Borthakur 提交于
Summary: There was an artifical limit on the size of the write buffer size. Test Plan: make check Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D10911
-
https://reviews.facebook.net/D10863由 Dhruba Borthakur 提交于
Summary: The valgrind errors were in the unit tests where we change the number of levels of a database using internal methods. Test Plan: valgrind ./reduce_levels_test valgrind ./db_test Reviewers: emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D10893
-
由 Haobo Xu 提交于
Summary: Make Statistics usable by client Test Plan: make check; db_bench Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D10899
-
- 22 5月, 2013 6 次提交
-
-
由 Vamsi Ponnekanti 提交于
Summary: This is initial version. A few ways in which this could be extended in the future are: (a) Killing from more places in source code (b) Hashing stack and using that hash in determining whether to crash. This is to avoid crashing more often at source lines that are executed more often. (c) Raising exceptions or returning errors instead of killing Test Plan: This whole thing is for testing. Here is part of output: python2.7 tools/db_crashtest2.py -d 600 Running db_stress db_stress retncode -15 output LevelDB version : 1.5 Number of threads : 32 Ops per thread : 10000000 Read percentage : 50 Write-buffer-size : 4194304 Delete percentage : 30 Max key : 1000 Ratio #ops/#keys : 320000 Num times DB reopens: 0 Batches/snapshots : 1 Purge redundant % : 50 Num keys per lock : 4 Compression : snappy ------------------------------------------------ No lock creation because test_batches_snapshots set 2013/04/26-17:55:17 Starting database operations Created bg thread 0x7fc1f07ff700 ... finished 60000 ops Running db_stress db_stress retncode -15 output LevelDB version : 1.5 Number of threads : 32 Ops per thread : 10000000 Read percentage : 50 Write-buffer-size : 4194304 Delete percentage : 30 Max key : 1000 Ratio #ops/#keys : 320000 Num times DB reopens: 0 Batches/snapshots : 1 Purge redundant % : 50 Num keys per lock : 4 Compression : snappy ------------------------------------------------ Created bg thread 0x7ff0137ff700 No lock creation because test_batches_snapshots set 2013/04/26-17:56:15 Starting database operations ... finished 90000 ops Revert Plan: OK Task ID: #2252691 Reviewers: dhruba, emayanke Reviewed By: emayanke CC: leveldb, haobo Differential Revision: https://reviews.facebook.net/D10581
-
由 Haobo Xu 提交于
Summary: Currently, with paranoid_check on, DB::Open will fail on any log read error on recovery. If client is ok with losing most recent updates, we could simply skip those errors. However, it's important to introduce an additional flag, so that paranoid_check can still guard against more serious problems. Test Plan: make check; db_stress Reviewers: dhruba, emayanke Reviewed By: emayanke CC: leveldb, emayanke Differential Revision: https://reviews.facebook.net/D10869
-
由 Dhruba Borthakur 提交于
Summary: There is an existing field Options.max_bytes_for_level_multiplier that sets the multiplier for the size of each level in the database. This patch introduces the ability to set different multipliers for every level in the database. The size of a level is determined by using both max_bytes_for_level_multiplier as well as the per-level fanout. size of level[i] = size of level[i-1] * max_bytes_for_level_multiplier * fanout[i-1] The default value of fanout is 1, so that it is backward compatible. Test Plan: make check Reviewers: haobo, emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D10863
-
由 Haobo Xu 提交于
Summary: The merge operator diff introduced a performance problem in MemTable::Get. An exit condition is missed when the current key does not match the user key. This could lead to full memtable scan if the user key is not found. Test Plan: make check; db_bench Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10851
-
由 Mayank Agarwal 提交于
Summary: db can't reopen safely with disable_wal set! Test Plan: make db_stress; run db_stress with disable_wal and reopens set and see error Reviewers: dhruba, vamsi Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10857
-
由 Haobo Xu 提交于
Summary: PosixLogger and AutoRollLogger do not seem to be thread safe. For PosixLogger, log_size_ is not atomically updated. For AutoRollLogger, the underlying logger_ might be deleted by one thread while still being accessed by another. Test Plan: make check Reviewers: kailiu, dhruba, heyongqiang Reviewed By: kailiu CC: leveldb, zshao, sheki Differential Revision: https://reviews.facebook.net/D9699
-
- 21 5月, 2013 1 次提交
-
-
由 Mayank Agarwal 提交于
Summary: Will help while debugging if the generated value is truncated at proper length. Test Plan: make db_stress;/db_stress --max_key=10000 --db=/tmp/mcr --threads=1 --ops_per_thread=10000 Reviewers: dhruba, vamsi Reviewed By: vamsi Differential Revision: https://reviews.facebook.net/D10845
-
- 18 5月, 2013 2 次提交
-
-
由 Haobo Xu 提交于
Summary: assert => ASSERT_TRUE Test Plan: make release; make check Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10839
-
由 Abhishek Kona 提交于
Summary: Make stop watch a simple implementation, instead of subclass of a virtual class Allocate stop watches off the stack instead of heap. Code is more terse now. Test Plan: make all check, db_bench with --statistics=1 Reviewers: haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D10809
-
- 17 5月, 2013 1 次提交
-
-
由 Abhishek Kona 提交于
Summary: Statistics.h and histogram.h had double based api's to record values. Remove them as they are not used anywhere Test Plan: make all check Reviewers: haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D10815
-
- 16 5月, 2013 1 次提交
-
-
由 Mayank Agarwal 提交于
Summary: ldb works with raw data from the database and needs to be aware of ttl-database to work with it meaningfully. '-ttl' option now tells it that. Also added onto the ldb_test.py test. This option may be specified alongwith put, get, scan or dump. There is no support to provide a ttl-value and it uses default forever because there is no use-case for this currently. Test Plan: make ldb_test; python tools/ldb_test.py Reviewers: dhruba, sheki, haobo, vamsi Reviewed By: sheki CC: leveldb Differential Revision: https://reviews.facebook.net/D10797
-
- 15 5月, 2013 1 次提交
-
-
由 Mayank Agarwal 提交于
Summary: This caused crash when the jenkins machine changed because the PATH variable is different. Test Plan: make Reviewers: dhruba, sheki Reviewed By: sheki CC: leveldb Differential Revision: https://reviews.facebook.net/D10803
-
- 14 5月, 2013 3 次提交
-
-
由 Deon Nicholas 提交于
Summary: Implemented the StringAppendOperator class (subclass of MergeOperator). Found in utilities/merge_operators/string_append/stringappend.{h,cc} It is a rocksdb Merge Operator that supports string/list concatenation with a configurable delimiter. The tests are found in .../stringappend_test.cc. It implements a map : key -> (list of strings), with core operations Append(list_key,val) and Get(list_key). Test Plan: 1. Navigate to your rocksdb repository 2. Execute: make stringappend_test (to compile) 3. Execute: ./stringappend_test (to run the tests) 4. Execute: make all check (to test the ENTIRE rocksdb codebase / regression) Reviewers: haobo, dhruba, zshao Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D10737
-
由 Haobo Xu 提交于
[RocksDB] Cleanup compaction filter to use a class interface, instead of function pointer and additional context pointer. Summary: This diff replaces compaction_filter_args and CompactionFilter with a single compaction_filter parameter. It gives CompactionFilter better encapsulation and a similar look to Comparator and MergeOpertor, which improves consistency of the overall interface. The change is not backward compatible. Nevertheless, the two references in fbcode are not in production yet. Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb, zshao Differential Revision: https://reviews.facebook.net/D10773
-
由 Haobo Xu 提交于
Summary: Currently, compaction filter is run on internal key older than the oldest snapshot, which is incorrect. Compaction filter should really be run on the most recent internal key when there is no external snapshot. Test Plan: make check; db_stress Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D10641
-
- 11 5月, 2013 3 次提交
-
-
由 Mayank Agarwal 提交于
Summary: added an argument to openttldb for read only and open the db in normal readonly mode if the arguments is set to true Test Plan: make ttl_test; ./ttl_test Reviewers: dhruba, haobo, vamsi, sheki Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10749
-
由 Abhishek Kona 提交于
Test Plan: make check Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10575
-
由 Abhishek Kona 提交于
Summary: Our ticker was not atomic. This was based on the assumption that we will increment counts at places only protected by Mutex. This is hard to program for and easy to make mistakes (mutex may be held a few layers above etc). Also this increases the instructions executed when the mutex is held. Test Plan: make check Reviewers: haobo, dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10569
-
- 10 5月, 2013 3 次提交
-
-
由 Mayank Agarwal 提交于
Summary: Since we are keeping 'leveldb' instead of 'rocksdb' in third-party, this is only logical. Test Plan: make clean;make Reviewers: sheki, dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D10719
-
由 Mayank Agarwal 提交于
Summary: The 2 checks added will increase reliabilty and help in debugging Test Plan: make ttl_test;./ttl_test Reviewers: vamsi, dhruba, sheki, haobo Reviewed By: vamsi Differential Revision: https://reviews.facebook.net/D10713
-
由 Mayank Agarwal 提交于
Summary: Will not be caught properly from file in fbcode with old path. New path fixes it. Test Plan: make Reviewers: sheki, dhruba, haobo, vamsi Reviewed By: sheki Differential Revision: https://reviews.facebook.net/D10707
-
- 09 5月, 2013 1 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: For level-0 compactions, we try to find if can include more L0 files in the same compaction run. This causes the 'smallest' and 'largest' key to get extended to a larger range. But the suceeding call to ParentRangeInCompaction() was still using the earlier values of 'smallest' and 'largest', Because of this bug, a file in L1 can be part of two concurrent compactions: one L0-L1 compaction and the other L1-L2 compaction. This should not cause any data loss, but will cause an assertion failure with debug builds. Test Plan: make check Differential Revision: https://reviews.facebook.net/D10677
-
- 07 5月, 2013 2 次提交
-
-
由 Abhishek Kona 提交于
Summary: WAL files are moved to archive directory and clear only at DB::Open. Can lead to a lot of space consumption in a Database. Added logic to periodically clear Archive Directory too. Test Plan: make all check + add unit test Reviewers: dhruba, heyongqiang Reviewed By: heyongqiang CC: leveldb Differential Revision: https://reviews.facebook.net/D10617
-
由 Haobo Xu 提交于
Summary: makefile change: LIBRARY => LIBOBJECTS thanks Abhishek for reproducing this locally. Test Plan: make release Reviewers: sheki CC: leveldb Task ID: # Blame Rev:
-
- 04 5月, 2013 2 次提交
-
-
由 Haobo Xu 提交于
Summary: This diff introduces a new Merge operation into rocksdb. The purpose of this review is mostly getting feedback from the team (everyone please) on the design. Please focus on the four files under include/leveldb/, as they spell the client visible interface change. include/leveldb/db.h include/leveldb/merge_operator.h include/leveldb/options.h include/leveldb/write_batch.h Please go over local/my_test.cc carefully, as it is a concerete use case. Please also review the impelmentation files to see if the straw man implementation makes sense. Note that, the diff does pass all make check and truly supports forward iterator over db and a version of Get that's based on iterator. Future work: - Integration with compaction - A raw Get implementation I am working on a wiki that explains the design and implementation choices, but coding comes just naturally and I think it might be a good idea to share the code earlier. The code is heavily commented. Test Plan: run all local tests Reviewers: dhruba, heyongqiang Reviewed By: dhruba CC: leveldb, zshao, sheki, emayanke, MarkCallaghan Differential Revision: https://reviews.facebook.net/D9651
-
由 Mayank Agarwal 提交于
Summary: value function in ttl-iterator was returning string which would have been freed before its usage as a slice. Thanks valgrind! Test Plan: valgrind ./ttl_test Reviewers: dhruba, haobo, sheki, vamsi Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D10635
-
- 03 5月, 2013 1 次提交
-
-
由 Mayank Agarwal 提交于
Summary: When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open. Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest Reviewed By: vamsi CC: zshao, xjin, vkrest, MarkCallaghan Differential Revision: https://reviews.facebook.net/D10311
-
- 30 4月, 2013 1 次提交
-
-
由 Abhishek Kona 提交于
Summary: The current code has a bug that take address of stack allocated LogReporter. It is causing SIGSEGV because the stack address is no longer valid when referenced. Test Plan: Tested on prod. Reviewers: haobo, dhruba, heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D10557
-