- 06 10月, 2015 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Refactor dump_tool and undump_tool so that it's possible to use them with customized options for example setting a specific comparator similar to what Dragon is doing with the LdbTool https://phabricator.fb.com/diffusion/FBCODE/browse/master/dragon/tools/Ldb.cpp Test Plan: compiles used it to dump / undump a dragon shard Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba, adsharma Differential Revision: https://reviews.facebook.net/D47853
-
- 29 9月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following tsan warning in db_stress.cc WARNING: ThreadSanitizer: data race (pid=3163194) Read of size 8 at 0x7fd1797cb518 by thread T32: #0 VerifyDb tools/db_stress.cc:1731 (db_stress+0x000000040674) #1 rocksdb::StressTest::ThreadBody(void*) tools/db_stress.cc:1191 (db_stress+0x0000000625a9) #2 StartThreadWrapper util/env_posix.cc:1648 (db_stress+0x00000028bbbd) Previous write of size 8 at 0x7fd1797cb518 by thread T31: #0 VerifyDb tools/db_stress.cc:1726 (db_stress+0x00000004072a) #1 rocksdb::StressTest::ThreadBody(void*) tools/db_stress.cc:1191 (db_stress+0x0000000625a9) #2 StartThreadWrapper util/env_posix.cc:1648 (db_stress+0x00000028bbbd) The cause is that in VerifyDb(), the static local const variable long max_key can be read and written at the same time. This patch fixed it by making it non-static. Test Plan: db_stress Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47703
-
- 18 9月, 2015 1 次提交
-
-
由 Andres Noetzli 提交于
Summary: This patch fixes #7460559. It introduces SingleDelete as a new database operation. This operation can be used to delete keys that were never overwritten (no put following another put of the same key). If an overwritten key is single deleted the behavior is undefined. Single deletion of a non-existent key has no effect but multiple consecutive single deletions are not allowed (see limitations). In contrast to the conventional Delete() operation, the deletion entry is removed along with the value when the two are lined up in a compaction. Note: The semantics are similar to @igor's prototype that allowed to have this behavior on the granularity of a column family ( https://reviews.facebook.net/D42093 ). This new patch, however, is more aggressive when it comes to removing tombstones: It removes the SingleDelete together with the value whenever there is no snapshot between them while the older patch only did this when the sequence number of the deletion was older than the earliest snapshot. Most of the complex additions are in the Compaction Iterator, all other changes should be relatively straightforward. The patch also includes basic support for single deletions in db_stress and db_bench. Limitations: - Not compatible with cuckoo hash tables - Single deletions cannot be used in combination with merges and normal deletions on the same key (other keys are not affected by this) - Consecutive single deletions are currently not allowed (and older version of this patch supported this so it could be resurrected if needed) Test Plan: make all check Reviewers: yhchiang, sdong, rven, anthony, yoshinorim, igor Reviewed By: igor Subscribers: maykov, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43179
-
- 15 9月, 2015 1 次提交
-
-
由 Dmitry Marakasov 提交于
-
- 09 9月, 2015 1 次提交
-
-
由 Amit Arya 提交于
Summary: Added tests for two LDBCommands namely i) ManifestDumpCommand and ii) ListColumnFamiliesCommand. + Minor fix in the sscanf formatter (along relace C cast with C++ cast) + replacing localtime with localtime_r which is thread safe. Test Plan: make all && ./tools/ldb_test.py Reviewers: anthony, igor, IslamAbdelRahman, kradhakrishnan, lgalanis, rven, sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D45819
-
- 29 8月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: Add ZSTD compression type. The same way as adding LZ4. Test Plan: run all tests. Generate files in db_bench. Make sure reads succeed. But the SST files cannot be opened in older versions. Also some other adhoc tests. Reviewers: rven, anthony, IslamAbdelRahman, kradhakrishnan, igor Reviewed By: igor Subscribers: MarkCallaghan, maykov, yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D45747
-
- 23 8月, 2015 1 次提交
-
-
由 Mark Callaghan 提交于
Summary: db_bench output now displays Percentile many times with --statistics after read IO latency histograms were added. So I only need the last one in the report output. Task ID: # Blame Rev: Test Plan: run run_flash_bench.sh Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D45093
-
- 22 8月, 2015 1 次提交
-
-
由 Ari Ekmekji 提交于
Summary: Up until this point we had DbOptions.num_subcompactions, but it is semantically more correct to call this max_subcompactions since we will schedule *up to* DbOptions.max_subcompactions smaller compactions at a time during a compaction job. I also added a --subcompactions option to db_bench Test Plan: make all make check Reviewers: sdong, igor, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D45069
-
- 21 8月, 2015 1 次提交
-
-
由 Mark Callaghan 提交于
Summary: Changes include: * don't sync-on-commit for single writer thread in readwhile... tests * make default block size 8kb rather than 4kb to avoid too small blocks after compression * use snappy instead of zlib to avoid stalls from compression latency * disable statistics * use bytes_per_sync=8M to reduce throughput loss on disk * use open_files=-1 to reduce mutex contention Task ID: # Blame Rev: Test Plan: run benchmark Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D44961
-
- 19 8月, 2015 1 次提交
-
-
由 Ari Ekmekji 提交于
Summary: In prepration for running multiple threads at the same time during a compaction job, this patch assigns each subcompaction its own state (instead of sharing the one global CompactionState). Each subcompaction then uses this state to update its statistics, keep track of its snapshots, etc. during the course of execution. Then at the end of all the executions the statistics are aggregated across the subcompactions so that the final result is the same as if only one larger compaction had run. Test Plan: ./db_test ./db_compaction_test ./compaction_job_test Reviewers: sdong, anthony, igor, noetzli, yhchiang Reviewed By: yhchiang Subscribers: MarkCallaghan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43239
-
- 12 8月, 2015 1 次提交
-
-
由 Andres Notzli 提交于
Summary: While working on single delete support for db_bench, I realized that db_bench/db_stress contain a bunch of duplicate code related to copmression and found some typos. This patch removes duplicate code, typos and a redundant #ifndef in internal_stats.cc. Test Plan: make db_stress && make db_bench && ./db_bench --benchmarks=compress,uncompress Reviewers: yhchiang, sdong, rven, anthony, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43965
-
- 05 8月, 2015 4 次提交
-
-
由 sdong 提交于
Summary: In a recent change, crash_test can put data under TEST_TMPDIR. However, the directory is not cleaned before running the test, which may cause unexpected results. Clean it. Test Plan: Run white and black box crash test against non-existing, or non-empty but not compactible DBs, and make sure it works as expected. Reviewers: kradhakrishnan, rven, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43515
-
由 sdong 提交于
Summary: Currently, whitebox crash test is not really executed, because the DB is destroyed after each crash. With this fix, in the first half of the time, DB will keep opening the crashed DB and continue from there. Test Plan: "make whitebox_crash_test" and see the same DB keeps crashing and being reopened. Reviewers: IslamAbdelRahman, yhchiang, rven, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43503
-
由 sdong 提交于
Summary: Currently crash_test only puts data under /tmp. It is less flexible if we want to cover different file systems or media. Make crash_test to appreciate TEST_TMPDIR so that users can run it against another file system. Test Plan: Run blackbox_crash_test and whitebox_crash_test with or without TEST_TMPDIR set and make sure DBs are put in the right place Reviewers: kradhakrishnan, yhchiang, rven, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43509
-
由 sdong 提交于
Summary: crash_test now only runs complicated options, multiple column families, prefix hash, frequently changing options, many compaction threads, etc. These options are good to cover new features but we loss coverage in most common use cases. Furthermore, by running only for multiple column families, we are not able to create LSM trees that are large enough to cover some stress cases. Make half of crash_test runs the simply tests: single column family, default mem table, one compaction thread, no change options. Test Plan: Run crash_test Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43461
-
- 01 8月, 2015 1 次提交
-
-
由 Andres Noetzli 提交于
Summary: Fixed typos. Test Plan: None Reviewers: igor, yhchiang, sdong, anthony, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43365
-
- 30 7月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: Crash tests are supposed to restart the same DB after crashing, but it is now opening a different DB. Fix it. It's probably a leftover of https://reviews.facebook.net/D17073 Test Plan: Run the test and make sure the same Db is opened. Reviewers: kradhakrishnan, rven, igor, IslamAbdelRahman, yhchiang, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43197
-
- 22 7月, 2015 1 次提交
-
-
由 Dmitri Smirnov 提交于
Add heap_test, merge_helper_test Fix uninitialized pointers in db_sanity_test that cause SIGSEV when DB::Open fails in case compression is not linked.
-
- 18 7月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: When we first started, max_background_flushes was 0 by default and compaction thread was executing flushes (since there was no flush thread). Then, we switched the default max_background_flushes to 1. However, we still support the case where there is no flush thread and flushes are done in compaction. This is making our code a bit more complicated. By not supporting this use-case we can make our code simpler. We have a special case that when you set max_background_flushes to 0, we schedule the flush to execute on the compaction thread. Test Plan: make check (there might be some unit tests that depend on this behavior) Reviewers: IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41931
-
- 15 7月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: This is failing our tsan tests. Our new behavior is to fail DB::Open() if the requested compression is not available. The easiest fix is to make ldb_test not depend on compression. Test Plan: python tools/ldb_test.py Reviewers: sdong, yhchiang, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42075
-
- 14 7月, 2015 3 次提交
-
-
由 Igor Canadi 提交于
Summary: This option is guarding the feature implemented 2 and a half years ago: D8991. The feature was enabled by default back then and has been running without issues. There is no reason why any client would turn this feature off. I found no reference in fbcode. Test Plan: none Reviewers: sdong, yhchiang, anthony, dhruba Reviewed By: dhruba Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42063
-
由 Islam AbdelRahman 提交于
Summary: Block reduce_levels_test in ROCKSDB_LITE as LDBCommand is not supported Test Plan: make reduce_levels_test -j64 OPT=-DROCKSDB_LITE make reduce_levels_test -j64 make check -j64 Reviewers: sdong, igor, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D41967
-
由 sdong 提交于
Summary: This helps Windows port to format their changes, as discussed. Might have formatted some other codes too becasue last 10 commits include more. Test Plan: Build it. Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D41961
-
- 02 7月, 2015 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Make RocksDb build and run on Windows to be functionally complete and performant. All existing test cases run with no regressions. Performance numbers are in the pull-request. Test plan: make all of the existing unit tests pass, obtain perf numbers. Co-authored-by: Praveen Rao praveensinghrao@outlook.com Co-authored-by: Sherlock Huang baihan.huang@gmail.com Co-authored-by: Alex Zinoviev alexander.zinoviev@me.com Co-authored-by: Dmitri Smirnov dmitrism@microsoft.com
-
- 27 6月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: as title Test Plan: make check Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40785
-
- 20 6月, 2015 1 次提交
-
-
由 Michael Callahan 提交于
Summary: Hack up rocksdb_dump and rocksdb_undump utilities to get this task rolling/promote discussion. Test Plan: Dump/undump databases recursively to see if nothing is lost. Reviewers: sdong, yhchiang, rven, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37269
-
- 06 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: EventListener::OnFlushCompleted() now passes a structure instead of a list of parameters. This minimizes the API change in the future. Test Plan: listener_test compact_files_test example/compact_files_example Reviewers: kradhakrishnan, sdong, IslamAbdelRahman, rven, igor Reviewed By: rven, igor Subscribers: IslamAbdelRahman, rven, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39543
-
- 03 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add EventListener::OnTableFileCreated(), which will be called when a table file is created. This patch is part of the EventLogger and EventListener integration. Test Plan: Augment existing test in db/listener_test.cc Reviewers: anthony, kradhakrishnan, rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38865
-
- 31 5月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed db_stress by correcting the verification of column family names in the Listener of db_stress Test Plan: db_stress Reviewers: igor, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39255
-
- 30 5月, 2015 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a compile warning in db_stress in NDEBUG mode. Test Plan: make OPT=-DNDEBUG db_stress Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39213
-
由 agiardullo 提交于
Summary: Optimistic transactions supporting begin/commit/rollback semantics. Currently relies on checking the memtable to determine if there are any collisions at commit time. Not yet implemented would be a way of enuring the memtable has some minimum amount of history so that we won't fail to commit when the memtable is empty. You should probably start with transaction.h to get an overview of what is currently supported. Test Plan: Added a new test, but still need to look into stress testing. Reviewers: yhchiang, igor, rven, sdong Reviewed By: sdong Subscribers: adamretter, MarkCallaghan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D33435
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed the following compile warning in db_stress: error: 'OnCompactionCompleted' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override] Test Plan: make db_stress Reviewers: sdong, igor, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39207
-
由 Yueh-Hsuan Chiang 提交于
Summary: Include EventListener in stress test. Test Plan: make blackbox_crash_test whitebox_crash_test Reviewers: anthony, igor, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39105
-
- 29 5月, 2015 1 次提交
-
-
由 agiardullo 提交于
Summary: For transactions, we are using the memtables to validate that there are no write conflicts. But after flushing, we don't have any memtables, and transactions could fail to commit. So we want to someone keep around some extra history to use for conflict checking. In addition, we want to provide a way to increase the size of this history if too many transactions fail to commit. After chatting with people, it seems like everyone prefers just using Memtables to store this history (instead of a separate history structure). It seems like the best place for this is abstracted inside the memtable_list. I decide to create a separate list in MemtableListVersion as using the same list complicated the flush/installalflushresults logic too much. This diff adds a new parameter to control how much memtable history to keep around after flushing. However, it sounds like people aren't too fond of adding new parameters. So I am making the default size of flushed+not-flushed memtables be set to max_write_buffers. This should not change the maximum amount of memory used, but make it more likely we're using closer the the limit. (We are now postponing deleting flushed memtables until the max_write_buffer limit is reached). So while we might use more memory on average, we are still obeying the limit set (and you could argue it's better to go ahead and use up memory now instead of waiting for a write stall to happen to test this limit). However, if people are opposed to this default behavior, we can easily set it to 0 and require this parameter be set in order to use transactions. Test Plan: Added a xfunc test to play around with setting different values of this parameter in all tests. Added testing in memtablelist_test and planning on adding more testing here. Reviewers: sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37443
-
- 23 5月, 2015 2 次提交
-
-
由 Aaron Schlesinger 提交于
-
由 Aaron Schlesinger 提交于
-
- 02 5月, 2015 1 次提交
-
-
由 Mark Callaghan 提交于
Summary: See https://gist.github.com/mdcallag/b867ee051d765760be0d for a sample Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37971
-
- 28 4月, 2015 1 次提交
-
-
由 Mark Callaghan 提交于
Summary: This runs a benchmark for LevelDB similar to what we have in tools/run_flash_bench.sh. It requires changes to db_bench that I published in a LevelDB fork on github. Some results are at: http://smalldatum.blogspot.com/2015/04/comparing-leveldb-and-rocksdb-take-2.html Sample output: ops/sec mb/sec usec/op avg p50 Test 525 16.4 1904.5 1904.5 111.0 fillseq.v32768 75187 15.5 13.3 13.3 4.4 fillseq.v200 28328 5.8 35.3 35.3 4.7 overwrite.t1.s0 175438 0.0 5.7 5.7 4.4 readrandom.t1 28490 5.9 35.1 35.1 4.7 overwrite.t1.s0 121951 0.0 8.2 8.2 5.7 readwhilewriting.t1 Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37749
-
- 26 4月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Build broken by https://github.com/facebook/rocksdb/commit/6ede020dc419c1621254f26060076ee6d2c2d792 Test Plan: make all Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37689
-
- 25 4月, 2015 1 次提交
-
-
由 clark.kang 提交于
-