- 25 4月, 2014 10 次提交
-
-
由 Igor Canadi 提交于
Summary: Now that we have column families involved, we need to add extra context to every log message. They now start with "[column family name] log message" Also added some logging that I think would be useful, like level summary after every flush (I often needed that when going through the logs). Test Plan: make check + ran db_bench to confirm I'm happy with log output Reviewers: dhruba, haobo, ljin, yhchiang, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18303
-
由 Igor Canadi 提交于
Summary: Now this gives us the real deal stack trace: Assertion failed: (false), function GetProperty, file db/db_impl.cc, line 4072. Received signal 6 (Abort trap: 6) #0 0x7fff57ce39b9 #1 abort (in libsystem_c.dylib) + 125 #2 basename (in libsystem_c.dylib) + 0 #3 rocksdb::DBImpl::GetProperty(rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) (in db_test) (db_impl.cc:4072) #4 rocksdb::_Test_Empty::_Run() (in db_test) (testharness.h:68) #5 rocksdb::_Test_Empty::_RunIt() (in db_test) (db_test.cc:1005) #6 rocksdb::test::RunAllTests() (in db_test) (testharness.cc:60) #7 main (in db_test) (db_test.cc:6697) #8 start (in libdyld.dylib) + 1 Test Plan: added artificial assert, saw great stack trace Reviewers: haobo, dhruba, ljin Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18309
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
[Java] Enable filluniquerandom, readseq, BloomFilter, and 70+ command-line options to DbBenchmark.java Summary: * Add filluniquerandom * Add readseq, implemented using iterator. * Realize most command-line-arguments from db_bench.cc (70+). * Some code are commented out as some of the options in Options not yet have Java bindings. * Add default option to DbBenchmark. * RocksDB will now take the ownership of all c++ raw-pointers from Options, which includes a c++ raw-pointer for Filter. Test Plan: ./jdb_bench.sh --db=/tmp/rocksjava-bench/db --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/tmp/rocksjava-bench/wal --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=filluniquerandom,readseq,readrandom --use_existing_db=0 --threads=4 Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18267
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: I'm getting lots of e-mails with CompactionInputErrorParanoid failing. Most recent example early morning today was: http://ci-builds.fb.com/job/rocksdb_valgrind/562/consoleFull I'm putting a stop to these e-mails. I investigated why the test is flakey and it turns out it's because of non-determinsim of compaction scheduling. If there is a compaction after the last flush, CorruptFile will corrupt the compacted file instead of file at level 0 (as it assumes). That makes `Check(9, 9)` fail big time. I also saw some errors with table file getting outputed to >= 1 levels instead of 0. Also fixed that. Test Plan: Ran corruption_test 100 times without a failure. Previously it usually failed at 10th occurrence. Reviewers: dhruba, haobo, ljin Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D18285
-
由 Igor Canadi 提交于
Summary: Sometimes, our tests fail because of normal `assert` call. It would be helpful to see stack trace in that case, too. Test Plan: Added `assert(false)` and verified it prints out stack trace Reviewers: haobo, dhruba, sdong, ljin, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18291
-
由 Igor Canadi 提交于
-
- 24 4月, 2014 4 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Currently, whenever DB Verification fails we bail out by calling `exit(1)`. This is kind of bad since it causes unclean shutdown and spew of error log messages like: 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument 05:03:27 pthread lock: Invalid argument This diff adds a new parameter that is set to true when verification fails. It can then use the parameter to bail out safely. Test Plan: Casued artificail failure. Verified that exit was clean. Reviewers: dhruba, haobo, ljin Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18243
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add Java bindings and Java tests for 30+ rocksdb::Options. Codes are machine-genearted based on include/rocksdb/options.h with manual-polishment. Test Plan: make rocksdbjava make jtest Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18273
-
由 sdong 提交于
Summary: IterKey set buffer_size_ to a wrong initial value, causing it to always allocate values from heap instead of stack if the key size is smaller. Fix it. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: haobo CC: igor, dhruba, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D18279
-
- 23 4月, 2014 13 次提交
-
-
由 Igor Canadi 提交于
Summary: While debugging Mac-only issue with ThreadLocalPtr, this was very useful. Let's print out stack trace in MAC OS, too. Test Plan: Verified that somewhat useful stack trace was generated on mac. Will run PrintStack() on linux, too. Reviewers: ljin, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18189
-
由 sdong 提交于
Summary: In this patch, two new DB properties are defined: rocksdb.num-immutable-mem-table and rocksdb.num-entries-imm-mem-tables, from where number of entries in mem tables can be exposed to users Test Plan: Cover the codes in db_test make all check Reviewers: haobo, ljin, igor Reviewed By: igor CC: nkg-, igor, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18207
-
由 Lei Jin 提交于
Summary: Get rid of the devil. Probably won't impact anything on the perf side. Test Plan: make all check Reviewers: igor, haobo, sdong, yhchiang Reviewed By: haobo CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D18153
-
由 Lei Jin 提交于
Summary: make singleton a static member instead of dynamic object. This should also avoid the race on unique_ptr Test Plan: make all check Reviewers: igor, haobo, sdong Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18177
-
由 sdong 提交于
Summary: This is a temp solution to expose index sizes to users from PlainTableReader before we persistent them to files. In this patch, the memory consumption of indexes used by PlainTableReader will be reported as two user defined properties, so that users can monitor them. Test Plan: Add a unit test. make all check` Reviewers: haobo, ljin Reviewed By: haobo CC: nkg-, yhchiang, igor, ljin, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18195
-
由 Igor Canadi 提交于
This reverts commit ddafceb6.
-
由 Igor Canadi 提交于
Summary: Using ThreadLocalPtr as a flag to determine if a mutex is locked or not enables us to implement AssertNotHeld(). It also makes AssertHeld() actually correct. I had to remove port::Mutex as a dependency for util/thread_local.h, but that's fine since we can just use std::mutex :) Test Plan: make check Reviewers: ljin, dhruba, haobo, sdong, yhchiang Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D18171
-
由 Yueh-Hsuan Chiang 提交于
[Java] Add bloom filter JNI bindings
-
-
由 Igor Canadi 提交于
Summary: This will enable people using TTL DB to do so with multiple column families. They can also specify different TTLs for each one. TODO: Implement CreateColumnFamily() in TTL world. Test Plan: Added a very simple sanity test. Reviewers: dhruba, haobo, ljin, sdong, yhchiang Reviewed By: haobo CC: leveldb, alberts Differential Revision: https://reviews.facebook.net/D17859
-
由 Ankit Gupta 提交于
-
由 James Pearce 提交于
-
由 Ankit Gupta 提交于
-
- 22 4月, 2014 13 次提交
-
-
由 Ankit Gupta 提交于
-
由 Ankit Gupta 提交于
-
由 Ankit Gupta 提交于
-
由 Ankit Gupta 提交于
-
由 Ankit Gupta 提交于
-
由 Ankit Gupta 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix the sigfault when running sst_dump on v2.6 db. Test Plan: git checkout bba6595b make clean make db_bench ./db_bench --db=/tmp/some/db --benchmarks=fillseq arc patch D18039 make clean make sst_dump ./sst_dump --file=/tmp/some/db --command=check Reviewers: igor, haobo, sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18039
-
由 Igor Canadi 提交于
Summary: Calling Fsync()/Sync() on a file should give the guarantee that whatever you written to the file is now persisted. This is currently not the case, since we might have some data left in application cache as we do Fsync()/Sync(). For example, BuildTable() calls Fsync() without the flush, assuming all sst data is now persisted, but it's actually not. This may result in big inconsistencies. Test Plan: no test Reviewers: sdong, dhruba, haobo, ljin, yhchiang Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18159
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add Java binding and test for rocksdb::ReadOptions. Test Plan: make rocksdbjava make jtest Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18129
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add Java bindings for memtables and sst format. Specifically, add two abstract Java classses --- MemTableConfig and SstFormatConfig. Each MemTable / SST implementation should has its own config class extends MemTableConfig / SstFormatConfig respectively and pass it to Options via setMemTableConfig / setSstConfig. Test Plan: make rocksdbjava make jdb_test make jdb_bench java/jdb_bench.sh \ --benchmarks=fillseq,readrandom,readwhilewriting \ --memtablerep=hash_skiplist \ --use_plain_table=1 \ --key_size=20 \ --prefix_size=12 \ --value_size=100 \ --cache_size=17179869184 \ --disable_wal=0 \ --sync=0 \ Reviewers: haobo, ankgup87, sdong Reviewed By: haobo CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D17997
-
由 Igor Canadi 提交于
Also, make `benchharness.cc` not compiled into rocksdb library.
-