- 10 5月, 2014 2 次提交
-
-
由 Dallas Marlow 提交于
-
由 Tyler Neely 提交于
-
- 08 5月, 2014 4 次提交
-
-
由 dallas marlow 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: This was reported by our customers in task #4295529. Cause: * MANIFEST file contains a VersionEdit, which contains file entries whose 'smallest' and 'largest' internal keys are empty. String with zero characters. Root cause of corruption was not investigated. We should report corruption when this happens. However, we currently SIGSEGV. Here's what happens: * VersionEdit encodes zero-strings happily and stores them in smallest and largest InternalKeys. InternalKey::Encode() does assert when `rep_.empty()`, but we don't assert in production environemnts. Also, we should never assert as a result of DB corruption. * As part of our ConsistencyCheck, we call GetLiveFilesMetaData() * GetLiveFilesMetadata() calls `file->largest.user_key().ToString()` * user_key() function does: 1. assert(size > 8) (ooops, no assert), 2. returns `Slice(internal_key.data(), internal_key.size() - 8)` * since `internal_key.size()` is unsigned int, this call translates to `Slice(whatever, 1298471928561892576182756)`. Bazinga. Fix: * VersionEdit checks if InternalKey is valid in `VersionEdit::GetInternalKey()`. If it's invalid, returns corruption. Lessons learned: * Always keep in mind that even if you `assert()`, production code will continue execution even if assert fails. * Never `assert` based on DB corruption. Assert only if the code should guarantee that assert can't fail. Test Plan: dumped offending manifest. Before: assert. Now: corruption Reviewers: dhruba, haobo, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18507
-
由 Igor Canadi 提交于
Summary: We have a lot of problems with gflags. However, when compiling rocksdb static library, we don't need gflags dependency. Reorganize INSTALL.md such that first-time customers don't need any dependency installed to actually build rocksdb static library. Test Plan: none Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18501
-
- 07 5月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: One of our users reported current file corruption. The machine was rebooted during the time. This is the only think I can think of which could cause current file corruption. Just add this paranoid check. Test Plan: make all check Reviewers: haobo, igor Reviewed By: haobo CC: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18495
-
- 06 5月, 2014 7 次提交
-
-
由 sdong 提交于
Summary: TLB page allocation errors are now logged to info logs, instead of stderr. In order to do that, mem table rep's factory functions take a info logger now. Test Plan: make all check Reviewers: haobo, igor, yhchiang Reviewed By: yhchiang CC: leveldb, yhchiang, dhruba Differential Revision: https://reviews.facebook.net/D18471
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: We need to lock accesses to some TestEnv variables. Otherwise we get failures like http://ci-builds.fb.com/job/rocksdb_asan_check/657/console Test Plan: make check Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18489
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: There are some projects in fbcode that define lz4 dependency on r108. We, however, defined dependency on r117. That produced some interesting issues and our build system was not happy. This diff makes rocksdb work with both r108 and r117. Hopefully this will fix our problems. Test Plan: compiled rocksdb with both r108 and r117 lz4 Reviewers: dhruba, sdong, haobo Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D18465
-
由 Igor Canadi 提交于
Summary: db_test includes Benchmark for LogAndApply. This diff removes it from db_test and puts it into a separate log_and_apply bench. I just wanted to play around with our new benchmark framework and figure out how it works. I would also like to show you how great it is! I believe right set of microbenchmarks can speed up our productivity a lot and help catch early regressions. Test Plan: no Reviewers: dhruba, haobo, sdong, ljin, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D18261
-
由 sdong 提交于
Summary: PlainTableFactory::PlainTableFactory() now has Huge TLB page feature turned on by default. Although it is not a public API (which we always turn the feature off now), our unit tests, like db_test sometimes uses it directly, which causes wrong coverage of codes. This patch fix it to allow unit tests to run with the correct setting Test Plan: Run db_test and make sure this feature is not on any more. Reviewers: igor, haobo Reviewed By: igor CC: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18483
-
- 05 5月, 2014 4 次提交
-
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
-
由 Igor Canadi 提交于
-
- 04 5月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Revert "Allow allocating dynamic bloom, plain table indexes and hash linked list from huge page TLB" This reverts commit 7dafa3a1.
-
- 03 5月, 2014 4 次提交
-
-
由 Benjamin Renard 提交于
Summary: added a new option to BackupEngine: if share_files_with_checksum is set to true, sst files are stored in shared_checksum/ and are identified by the triple (file name, checksum, file size) instead of just the file name. This option is targeted at distributed databases that want to backup their primary replica. Test Plan: unit tests and tested backup and restore on a distributed rocksdb Reviewers: igor Reviewed By: igor Differential Revision: https://reviews.facebook.net/D18393
-
由 Igor Canadi 提交于
Summary: link our fbcode build with lz4 Test Plan: compiles Reviewers: sdong, haobo, dhruba, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D18447
-
由 Igor Canadi 提交于
Summary: Added a method that executes a callback on every cache entry. Test Plan: added a unit test Reviewers: haobo Reviewed By: haobo CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D18441
-
由 Igor Canadi 提交于
-
- 02 5月, 2014 5 次提交
-
-
由 Igor Canadi 提交于
Summary: Our valgrind tests are failing because ttl_test is kind of flakey. This diff should fix valgrind issue and make ttl_test less flakey and much faster. Instead of relying on Env::Default() for getting current time, I expose `Env*` to all TTL functions that are interested in time. That way, I can insert a custom test Env which is then used to provide exactly the times we need. That way, we don't need to sleep anymore -- we control the time. Test Plan: ttl_test in normal and valgrind run Reviewers: dhruba, haobo, sdong, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18399
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Originally: https://github.com/facebook/rocksdb/pull/87/files I'm taking over to apply some finishing touches Test Plan: will add tests Reviewers: dhruba, haobo, sdong, yhchiang, ljin Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D18315
-
- 01 5月, 2014 7 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add RocksObject, the base class of all java objects which has a c++ pointer. While the finalizer of a RocksObject will release its c++ resource, it is suggested to call its RocksObject.dispose() to manually release its c++ resource. Existing RocksDB java classes are now extending RocksObject. Test Plan: make rocksdbjava make jtest make jdb_bench Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18411
-
由 Igor Canadi 提交于
Summary: As summary Test Plan: compiles :) Reviewers: dhruba, haobo, sdong, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18405
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add static method RocksDB.loadLibrary() which loads necessary library files. Test Plan: make rocksdbjava make jtest make jdb_bench java/jdb_bench.sh Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett Reviewed By: ankgup87 CC: leveldb Differential Revision: https://reviews.facebook.net/D18375
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Added a new option `max_total_wal_size`. Once the total WAL size goes over that, we make an attempt to flush all column families that still have data in the earliest WAL file. By default, I calculate `max_total_wal_size` dynamically, that should be good-enough for non-advanced customers. Test Plan: Added a test Reviewers: dhruba, haobo, sdong, ljin, yhchiang Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D18345
-
由 sdong 提交于
Summary: Add an option to allocate a piece of memory from huge page TLB. Add options to trigger it in dynamic bloom, plain table indexes andhash linked list hash table. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: haobo CC: nkg-, dhruba, leveldb, igor, yhchiang Differential Revision: https://reviews.facebook.net/D18357
-
由 Igor Canadi 提交于
-
- 30 4月, 2014 5 次提交
-
-
由 Igor Canadi 提交于
-
由 Yueh-Hsuan Chiang 提交于
[Java] Add multiget JNI bindings
-
由 Igor Canadi 提交于
Improve building of shared library so that it e.g. works with Go bindings
-
由 Yueh-Hsuan Chiang 提交于
Summary: = Major Changes = * Add a new mem-table representation, HashCuckooRep, which is based cuckoo hash. Cuckoo hash uses multiple hash functions. This allows each key to have multiple possible locations in the mem-table. - Put: When insert a key, it will try to find whether one of its possible locations is vacant and store the key. If none of its possible locations are available, then it will kick out a victim key and store at that location. The kicked-out victim key will then be stored at a vacant space of its possible locations or kick-out another victim. In this diff, the kick-out path (known as cuckoo-path) is found using BFS, which guarantees to be the shortest. - Get: Simply tries all possible locations of a key --- this guarantees worst-case constant time complexity. - Time complexity: O(1) for Get, and average O(1) for Put if the fullness of the mem-table is below 80%. - Default using two hash functions, the number of hash functions used by the cuckoo-hash may dynamically increase if it fails to find a short-enough kick-out path. - Currently, HashCuckooRep does not support iteration and snapshots, as our current main purpose of this is to optimize point access. = Minor Changes = * Add IsSnapshotSupported() to DB to indicate whether the current DB supports snapshots. If it returns false, then DB::GetSnapshot() will always return nullptr. Test Plan: Run existing tests. Will develop a test specifically for cuckoo hash in the next diff. Reviewers: sdong, haobo Reviewed By: sdong CC: leveldb, dhruba, igor Differential Revision: https://reviews.facebook.net/D16155
-
由 Krzysztof Kowalczyk 提交于
-