- 14 5月, 2014 7 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 sdong 提交于
Summary: As title Test Plan: make all check. Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18705
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a bug in Java DB Benchmark where random reads does not consider full key range. Test Plan: make rocksdbjava make jdb_bench cd java jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=fillseq --use_existing_db=false --num=100000 jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=readrandom --use_existing_db=true --num=100000 --reads=1000000 Reviewers: haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18693
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make read benchmarks print out found / not-found information. Test Plan: make rocksdbjava make jdb_bench cd java ./jdb_bench.sh Reviewers: haobo, sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18699
-
由 Igor Canadi 提交于
Summary: This diff addresses task #4296714 and rethinks how users provide us with TablePropertiesCollectors as part of Options. Here's description of task #4296714: I'm debugging #4295529 and noticed that our count of user properties kDeletedKeys is wrong. We're sharing one single InternalKeyPropertiesCollector with all Table Builders. In LOG Files, we're outputting number of kDeletedKeys as connected with a single table, while it's actually the total count of deleted keys since creation of the DB. For example, this table has 3155 entries and 1391828 deleted keys. The problem with current approach that we call methods on a single TablePropertiesCollector for all the tables we create. Even worse, we could do it from multiple threads at the same time and TablePropertiesCollector has no way of knowing which table we're calling it for. Good part: Looks like nobody inside Facebook is using Options::table_properties_collectors. This means we should be able to painfully change the API. In this change, I introduce TablePropertiesCollectorFactory. For every table we create, we call `CreateTablePropertiesCollector`, which creates a TablePropertiesCollector for a single table. We then use it sequentially from a single thread, which means it doesn't have to be thread-safe. Test Plan: Added a test in table_properties_collector_test that fails on master (build two tables, assert that kDeletedKeys count is correct for the second one). Also, all other tests Reviewers: sdong, dhruba, haobo, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D18579
-
由 Yueh-Hsuan Chiang 提交于
Summary: Before the Java binding for Env is ready, Java developers have no way to control the number of background threads. This diff provides a temporary solution where RocksDB.setMaxBackgroundCompactions() will affect the number of background threads. Note that once Env is ready. Changes made in this diff should be reverted. Test Plan: make rocksdbjava make jtest make jdb_bench java/jdb_bench.sh Reviewers: haobo, sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18681
-
- 13 5月, 2014 3 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a file-not-found issue when a log file is moved to archive by doing a missing retry. Test Plan: make db_test export ROCKSDB_TEST=TransactionLogIteratorRace ./db_test Reviewers: sdong, haobo Reviewed By: sdong CC: igor, leveldb Differential Revision: https://reviews.facebook.net/D18669
-
由 Yueh-Hsuan Chiang 提交于
Summary: Renamed Iterator to RocksIterator to avoid potential confliction with the Java built-in Iterator. Test Plan: make rocksdbjava make jtest Reviewers: haobo, dhruba, sdong, ankgup87 Reviewed By: sdong CC: leveldb, rsumbaly, swapnilghike, zzbennett Differential Revision: https://reviews.facebook.net/D18615
-
由 Igor Canadi 提交于
Fix building RocksDB in paths containing spaces
-
- 12 5月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
only use MAP_HUGETLB when the kernel supports it
-
- 11 5月, 2014 4 次提交
-
-
由 Dallas Marlow 提交于
-
由 Mike Lin 提交于
Fix building RocksDB in paths containing spaces -- quote path names in Makefile and build_detect_platform.
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: This diff is addressing multiple things with a single goal -- to make RocksDB easier to use: * Add some functions to Options that make RocksDB easier to tune. * Add example code for both simple RocksDB and RocksDB with Column Families. * Rewrite our README.md Regarding Options, I took a stab at something we talked about for a long time: * https://www.facebook.com/groups/rocksdb.dev/permalink/563169950448190/ I added functions: * IncreaseParallelism() -- easy, increases the thread pool and max_background_compactions * OptimizeLevelStyleCompaction(memtable_memory_budget) -- the easiest way to optimize rocksdb for less stalls with level style compaction. This is very likely not ideal configuration. Feel free to suggest improvements. I used some of Mark's suggestions from here: https://github.com/facebook/rocksdb/issues/54 * OptimizeUniversalStyleCompaction(memtable_memory_budget) -- optimize for universal compaction. Test Plan: compiled rocksdb. ran examples. Reviewers: dhruba, MarkCallaghan, haobo, sdong, yhchiang Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18621
-
- 10 5月, 2014 3 次提交
-
-
由 Dallas Marlow 提交于
-
由 sdong 提交于
Summary: as title Test Plan: Still compile Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18603
-
由 Tyler Neely 提交于
-
- 09 5月, 2014 6 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Newer gflags switched from `google` namespace to `gflags` namespace. See: https://github.com/facebook/rocksdb/issues/139 and https://github.com/facebook/rocksdb/issues/102 Unfortunately, they don't define any macro with their namespace, so we need to actually try to compile gflags with two different namespace to figure out which one is the correct one. Test Plan: works in fbcode environemnt. I'll also try in ubutnu with newer gflags Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18537
-
由 sdong 提交于
Summary: Use autovector in MergingIterator so that if there are 4 or less child iterators in it, iterator wrappers are inline, which is more likely to be cache friendly. Based on one test run with a shadow traffic of one product, it reduces CPU of MergingIterator::Seek() by half. Test Plan: make all check Reviewers: haobo, yhchiang, igor, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18531
-
由 Igor Canadi 提交于
Summary: Resize the autovector! Test Plan: test Reviewers: sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18543
-
由 Igor Canadi 提交于
Summary: We had a hypothesis in https://reviews.facebook.net/D18507 that empty-string internal keys might have been caused by compaction filter deleting all the entries. I added a unit test for that case. Unforutnately, everything works as expected. Test Plan: this is a test Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18519
-
由 sdong 提交于
Summary: This variable is not used. Remove it. Test Plan: build. Reviewers: haobo, igor, yhchiang Reviewed By: haobo CC: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18525
-
- 08 5月, 2014 4 次提交
-
-
由 dallas marlow 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: This was reported by our customers in task #4295529. Cause: * MANIFEST file contains a VersionEdit, which contains file entries whose 'smallest' and 'largest' internal keys are empty. String with zero characters. Root cause of corruption was not investigated. We should report corruption when this happens. However, we currently SIGSEGV. Here's what happens: * VersionEdit encodes zero-strings happily and stores them in smallest and largest InternalKeys. InternalKey::Encode() does assert when `rep_.empty()`, but we don't assert in production environemnts. Also, we should never assert as a result of DB corruption. * As part of our ConsistencyCheck, we call GetLiveFilesMetaData() * GetLiveFilesMetadata() calls `file->largest.user_key().ToString()` * user_key() function does: 1. assert(size > 8) (ooops, no assert), 2. returns `Slice(internal_key.data(), internal_key.size() - 8)` * since `internal_key.size()` is unsigned int, this call translates to `Slice(whatever, 1298471928561892576182756)`. Bazinga. Fix: * VersionEdit checks if InternalKey is valid in `VersionEdit::GetInternalKey()`. If it's invalid, returns corruption. Lessons learned: * Always keep in mind that even if you `assert()`, production code will continue execution even if assert fails. * Never `assert` based on DB corruption. Assert only if the code should guarantee that assert can't fail. Test Plan: dumped offending manifest. Before: assert. Now: corruption Reviewers: dhruba, haobo, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18507
-
由 Igor Canadi 提交于
Summary: We have a lot of problems with gflags. However, when compiling rocksdb static library, we don't need gflags dependency. Reorganize INSTALL.md such that first-time customers don't need any dependency installed to actually build rocksdb static library. Test Plan: none Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18501
-
- 07 5月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: One of our users reported current file corruption. The machine was rebooted during the time. This is the only think I can think of which could cause current file corruption. Just add this paranoid check. Test Plan: make all check Reviewers: haobo, igor Reviewed By: haobo CC: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18495
-
- 06 5月, 2014 7 次提交
-
-
由 sdong 提交于
Summary: TLB page allocation errors are now logged to info logs, instead of stderr. In order to do that, mem table rep's factory functions take a info logger now. Test Plan: make all check Reviewers: haobo, igor, yhchiang Reviewed By: yhchiang CC: leveldb, yhchiang, dhruba Differential Revision: https://reviews.facebook.net/D18471
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: We need to lock accesses to some TestEnv variables. Otherwise we get failures like http://ci-builds.fb.com/job/rocksdb_asan_check/657/console Test Plan: make check Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18489
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: There are some projects in fbcode that define lz4 dependency on r108. We, however, defined dependency on r117. That produced some interesting issues and our build system was not happy. This diff makes rocksdb work with both r108 and r117. Hopefully this will fix our problems. Test Plan: compiled rocksdb with both r108 and r117 lz4 Reviewers: dhruba, sdong, haobo Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18465
-
由 Igor Canadi 提交于
Summary: db_test includes Benchmark for LogAndApply. This diff removes it from db_test and puts it into a separate log_and_apply bench. I just wanted to play around with our new benchmark framework and figure out how it works. I would also like to show you how great it is! I believe right set of microbenchmarks can speed up our productivity a lot and help catch early regressions. Test Plan: no Reviewers: dhruba, haobo, sdong, ljin, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D18261
-
由 sdong 提交于
Summary: PlainTableFactory::PlainTableFactory() now has Huge TLB page feature turned on by default. Although it is not a public API (which we always turn the feature off now), our unit tests, like db_test sometimes uses it directly, which causes wrong coverage of codes. This patch fix it to allow unit tests to run with the correct setting Test Plan: Run db_test and make sure this feature is not on any more. Reviewers: igor, haobo Reviewed By: igor CC: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18483
-
- 05 5月, 2014 4 次提交
-
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 Igor Canadi 提交于
-