- 20 5月, 2014 1 次提交
-
-
由 sdong 提交于
ThreadPool to allow decrease number of threads and increase of number of threads is to be instantly scheduled Summary: Add a feature to decrease the number of threads in thread pool. Also instantly schedule more threads if number of threads is increased. Here is the way it is implemented: each background thread needs its thread ID. After decreasing number of threads, all threads are woken up. The thread with the largest thread ID will terminate. If there are more threads to terminate, the thread will wake up all threads again. Another change is made so that when number of threads is increased, more threads are created and all previous excessive threads are woken up to do the work. Test Plan: Add a unit test. Reviewers: haobo, dhruba Reviewed By: haobo CC: yhchiang, igor, nkg-, leveldb Differential Revision: https://reviews.facebook.net/D18675
-
- 17 5月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
-
- 16 5月, 2014 4 次提交
-
-
由 Igor Canadi 提交于
Summary: This diff adds rapidjson (https://code.google.com/p/rapidjson/) to RocksDB repository. First step to adding JSON is to add a JSON parser :) I'm not sure if rapidjson is the right choice. I only considered folly as an alternative. Using folly JSON parser has serious downsides. I tried extracting folly::json from the folly library. However, it depends on folly::dynamic, which basically depends on everything else. I would prefer to avoid adding complete folly library as RocksDB dependency. Folly has a lot of its own dependencies (https://github.com/facebook/folly) and also looks like open source world has some trouble installing it (https://groups.google.com/forum/#!forum/facebook-folly -- 60% of the posts are compile errors). We can discuss this if you think otherwise. RapidJSON does not have any dependencies whatsoever. No boost, no STL. We don't need to compile it since it's all header files. The performance results are also impressive: https://code.google.com/p/rapidjson/wiki/Performance. However, I'll make sure to write code in a way that we can easily switch JSON implementations going forward. Quora thread has some alternatives: http://www.quora.com/What-is-the-best-C-JSON-library Test Plan: none Reviewers: dhruba, haobo, yhchiang, sdong, jamesgpearce Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18729
-
由 Kai Liu 提交于
Summary: Materialize the hash index to avoid the soaring cpu/flash usage when initializing the database. Test Plan: existing unit tests passed Reviewers: sdong, haobo Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18339
-
由 sdong 提交于
Summary: Key size limit doesn't seem to be applicable anymore. Remove it. Test Plan: run a couple of tests in db_bench Reviewers: haobo, igor, yhchiang, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18723
-
由 Igor Canadi 提交于
-
- 15 5月, 2014 11 次提交
-
-
由 Igor Canadi 提交于
Fix errors while building with clang (against libc++) under linux
-
由 Chilledheart 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
This reverts commit 35a8873a.
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Copy improvements from fbcode's version of EnvHdfs to our open-source version. Some very important bug fixes in there. Test Plan: compiles Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18711
-
由 Igor Canadi 提交于
Summary: Cleaned up compaction logging a little bit. Now file sizes are easier to read. Also, removed the trailing space. Test Plan: verified that i'm happy with logging output: files_size[#33(seq=101,sz=98KB,0) #31(seq=81,sz=159KB,0) #26(seq=0,sz=637KB,0)] Reviewers: sdong, haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18549
-
由 sdong 提交于
Summary: In order to use arena to a use case that the total allocation size might be small (LogBuffer is already such a case), inline 1KB of data in it, so that it can be mostly in stack or inline in another class. If always inlining 2KB is a concern, I could make it a template to determine what to inline. However, dependents need to changes. Doesn't go with it for now Test Plan: make all check. Reviewers: haobo, igor, yhchiang, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18609
-
由 Igor Canadi 提交于
Travis CI
-
- 14 5月, 2014 7 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 sdong 提交于
Summary: As title Test Plan: make all check. Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18705
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a bug in Java DB Benchmark where random reads does not consider full key range. Test Plan: make rocksdbjava make jdb_bench cd java jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=fillseq --use_existing_db=false --num=100000 jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=readrandom --use_existing_db=true --num=100000 --reads=1000000 Reviewers: haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18693
-
由 Yueh-Hsuan Chiang 提交于
Summary: Make read benchmarks print out found / not-found information. Test Plan: make rocksdbjava make jdb_bench cd java ./jdb_bench.sh Reviewers: haobo, sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18699
-
由 Igor Canadi 提交于
Summary: This diff addresses task #4296714 and rethinks how users provide us with TablePropertiesCollectors as part of Options. Here's description of task #4296714: I'm debugging #4295529 and noticed that our count of user properties kDeletedKeys is wrong. We're sharing one single InternalKeyPropertiesCollector with all Table Builders. In LOG Files, we're outputting number of kDeletedKeys as connected with a single table, while it's actually the total count of deleted keys since creation of the DB. For example, this table has 3155 entries and 1391828 deleted keys. The problem with current approach that we call methods on a single TablePropertiesCollector for all the tables we create. Even worse, we could do it from multiple threads at the same time and TablePropertiesCollector has no way of knowing which table we're calling it for. Good part: Looks like nobody inside Facebook is using Options::table_properties_collectors. This means we should be able to painfully change the API. In this change, I introduce TablePropertiesCollectorFactory. For every table we create, we call `CreateTablePropertiesCollector`, which creates a TablePropertiesCollector for a single table. We then use it sequentially from a single thread, which means it doesn't have to be thread-safe. Test Plan: Added a test in table_properties_collector_test that fails on master (build two tables, assert that kDeletedKeys count is correct for the second one). Also, all other tests Reviewers: sdong, dhruba, haobo, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D18579
-
由 Yueh-Hsuan Chiang 提交于
Summary: Before the Java binding for Env is ready, Java developers have no way to control the number of background threads. This diff provides a temporary solution where RocksDB.setMaxBackgroundCompactions() will affect the number of background threads. Note that once Env is ready. Changes made in this diff should be reverted. Test Plan: make rocksdbjava make jtest make jdb_bench java/jdb_bench.sh Reviewers: haobo, sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18681
-
- 13 5月, 2014 3 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a file-not-found issue when a log file is moved to archive by doing a missing retry. Test Plan: make db_test export ROCKSDB_TEST=TransactionLogIteratorRace ./db_test Reviewers: sdong, haobo Reviewed By: sdong CC: igor, leveldb Differential Revision: https://reviews.facebook.net/D18669
-
由 Yueh-Hsuan Chiang 提交于
Summary: Renamed Iterator to RocksIterator to avoid potential confliction with the Java built-in Iterator. Test Plan: make rocksdbjava make jtest Reviewers: haobo, dhruba, sdong, ankgup87 Reviewed By: sdong CC: leveldb, rsumbaly, swapnilghike, zzbennett Differential Revision: https://reviews.facebook.net/D18615
-
由 Igor Canadi 提交于
Fix building RocksDB in paths containing spaces
-
- 12 5月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
only use MAP_HUGETLB when the kernel supports it
-
- 11 5月, 2014 5 次提交
-
-
由 Dallas Marlow 提交于
-
由 Mike Lin 提交于
Add a minimal .travis.yml for Travis CI. Some ugly hacks needed to get RocksDB building in Travis' OpenVZ Ubuntu 12.04 environment.
-
由 Mike Lin 提交于
Fix building RocksDB in paths containing spaces -- quote path names in Makefile and build_detect_platform.
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: This diff is addressing multiple things with a single goal -- to make RocksDB easier to use: * Add some functions to Options that make RocksDB easier to tune. * Add example code for both simple RocksDB and RocksDB with Column Families. * Rewrite our README.md Regarding Options, I took a stab at something we talked about for a long time: * https://www.facebook.com/groups/rocksdb.dev/permalink/563169950448190/ I added functions: * IncreaseParallelism() -- easy, increases the thread pool and max_background_compactions * OptimizeLevelStyleCompaction(memtable_memory_budget) -- the easiest way to optimize rocksdb for less stalls with level style compaction. This is very likely not ideal configuration. Feel free to suggest improvements. I used some of Mark's suggestions from here: https://github.com/facebook/rocksdb/issues/54 * OptimizeUniversalStyleCompaction(memtable_memory_budget) -- optimize for universal compaction. Test Plan: compiled rocksdb. ran examples. Reviewers: dhruba, MarkCallaghan, haobo, sdong, yhchiang Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18621
-
- 10 5月, 2014 3 次提交
-
-
由 Dallas Marlow 提交于
-
由 sdong 提交于
Summary: as title Test Plan: Still compile Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18603
-
由 Tyler Neely 提交于
-
- 09 5月, 2014 4 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Newer gflags switched from `google` namespace to `gflags` namespace. See: https://github.com/facebook/rocksdb/issues/139 and https://github.com/facebook/rocksdb/issues/102 Unfortunately, they don't define any macro with their namespace, so we need to actually try to compile gflags with two different namespace to figure out which one is the correct one. Test Plan: works in fbcode environemnt. I'll also try in ubutnu with newer gflags Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18537
-
由 sdong 提交于
Summary: Use autovector in MergingIterator so that if there are 4 or less child iterators in it, iterator wrappers are inline, which is more likely to be cache friendly. Based on one test run with a shadow traffic of one product, it reduces CPU of MergingIterator::Seek() by half. Test Plan: make all check Reviewers: haobo, yhchiang, igor, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D18531
-
由 Igor Canadi 提交于
Summary: Resize the autovector! Test Plan: test Reviewers: sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18543
-