- 17 6月, 2016 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Allow arcanist_util to work with both new and old arc versions. The diff is based on Adam Retter's pull request https://github.com/facebook/rocksdb/pull/1168 Many thanks to Adam to initiate this work Test Plan: run arc lint and arc diff using different arc versions Reviewers: sdong, IslamAbdelRahman, kradhakrishnan, andrewkr, adamretter, igor Reviewed By: adamretter Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59697
-
由 Islam AbdelRahman 提交于
Summary: We dont report the bytes that we ingested from AddFile which make the write amplification numbers incorrect Update InternalStats and add logging for AddFile() Test Plan: Make sure the code compile and existing tests pass Reviewers: lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59763
-
- 16 6月, 2016 3 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Travis is broken on linux, disable it temporarily Test Plan: https://travis-ci.org/facebook/rocksdb/builds/137721712 Reviewers: yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59673
-
由 sdong 提交于
Summary: DBCompactionTest.SkipStatsUpdateTest sometimes fails. I don't see any verification related to the deletes issued. Remove them to avoid the uncertainty. Test Plan: Run the test. Reviewers: IslamAbdelRahman, andrewkr, yhchiang Reviewed By: yhchiang Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59613
-
由 krad 提交于
Summary: Making access credentials for sandcastle configurable Test Plan: Submit for review Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59649
-
- 15 6月, 2016 6 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Fix clang build Test Plan: USE_CLANG=1 make check -j64 Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59667
-
由 sdong 提交于
Summary: char is not signed in some platforms. Having negative values confuse those compilers. Test Plan: Run all existing tests. Reviewers: andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59619
-
由 Islam AbdelRahman 提交于
Summary: Right now the way we do BlockIter::Prev() is like this - Go to the beginning of the restart interval - Keep moving forward (and decoding keys using ParseNextKey()) until we reach the desired key This can be optimized by caching the decoded entries in the first pass and reusing them in consecutive BlockIter::Prev() calls Before caching ``` DEBUG_LEVEL=0 make db_bench -j64 && ./db_bench --benchmarks="readreverse" --db="/dev/shm/bench_prev_opt/" --use_existing_db --disable_auto_compactions DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.413 micros/op 2423972 ops/sec; 268.2 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.414 micros/op 2413867 ops/sec; 267.0 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.410 micros/op 2440881 ops/sec; 270.0 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.414 micros/op 2417298 ops/sec; 267.4 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.413 micros/op 2421682 ops/sec; 267.9 MB/s ``` After caching ``` DEBUG_LEVEL=0 make db_bench -j64 && ./db_bench --benchmarks="readreverse" --db="/dev/shm/bench_prev_opt/" --use_existing_db --disable_auto_compactions DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.324 micros/op 3088955 ops/sec; 341.7 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.335 micros/op 2980999 ops/sec; 329.8 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.341 micros/op 2929681 ops/sec; 324.1 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.344 micros/op 2908490 ops/sec; 321.8 MB/s DB path: [/dev/shm/bench_prev_opt/] readreverse : 0.338 micros/op 2958404 ops/sec; 327.3 MB/s ``` Test Plan: COMPILE_WITH_ASAN=1 make check -j64 Reviewers: andrewkr, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D59463
-
由 krad 提交于
-
由 Islam AbdelRahman 提交于
Summary: compile and link with jemalloc in examples binaries Test Plan: cd examples make -j64 Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D59457
-
由 Adam Retter 提交于
* Java API - Expose SetSavePoint and RollbackToRestorePoint for WriteBatch and WriteBatchWithIndex * Minor cleanup
-
- 14 6月, 2016 6 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: [ This diff is stacked on top of D59577 ] We keep calling timer.ElapsedNanos() on every call to MergeOperator::FullMerge even when statistics are disabled, this is wasteful. I run the readseq benchmark on a DB containing 100K merge operands for 100K keys (1 operand per key) with 1GB block cache I see slight performance improvment Original results ``` $ ./db_bench --benchmarks="readseq,readseq,readseq,readseq,readseq,readseq,readseq,readseq,readseq" --merge_operator="max" --merge_keys=100000 --num=100000 --db="/dev/shm/100K_merge_compacted/" --cache_size=1073741824 --use_existing_db --disable_auto_compactions ------------------------------------------------ DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.498 micros/op 2006597 ops/sec; 222.0 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.295 micros/op 3393627 ops/sec; 375.4 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.285 micros/op 3511155 ops/sec; 388.4 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.286 micros/op 3500470 ops/sec; 387.2 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.283 micros/op 3530751 ops/sec; 390.6 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.289 micros/op 3464811 ops/sec; 383.3 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.277 micros/op 3612814 ops/sec; 399.7 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.283 micros/op 3539640 ops/sec; 391.6 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.285 micros/op 3503766 ops/sec; 387.6 MB/s ``` After patch ``` $ ./db_bench --benchmarks="readseq,readseq,readseq,readseq,readseq,readseq,readseq,readseq,readseq" --merge_operator="max" --merge_keys=100000 --num=100000 --db="/dev/shm/100K_merge_compacted/" --cache_size=1073741824 --use_existing_db --disable_auto_compactions ------------------------------------------------ DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.476 micros/op 2100119 ops/sec; 232.3 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.278 micros/op 3600887 ops/sec; 398.4 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.275 micros/op 3636698 ops/sec; 402.3 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.271 micros/op 3691661 ops/sec; 408.4 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.273 micros/op 3661534 ops/sec; 405.1 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.276 micros/op 3627106 ops/sec; 401.3 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.272 micros/op 3682635 ops/sec; 407.4 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.266 micros/op 3758331 ops/sec; 415.8 MB/s DB path: [/dev/shm/100K_merge_compacted/] readseq : 0.266 micros/op 3761907 ops/sec; 416.2 MB/s ``` Test Plan: make check -j64 Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59583
-
由 Gunnar Kudrjavets 提交于
Summary: Two changes here: - Remove dead Jenkins related code which is no longer relevant. - Support `arc diff --preview`. Currently it doesn't work because a step which applies a diff assumes that a revision has been created. Which in case of `--preview` isn't. Therefore diff can't be applied and validation fails. Solution is to use `--nocommit` because for validation purposes performing a commit isn't necessary. Test Plan: - Current changes are submitted using `arc diff --preview`. - All the pre-commit verification tests passed. Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: leveldb, andrewkr, jtolmer, dhruba Differential Revision: https://reviews.facebook.net/D59571
-
由 Islam AbdelRahman 提交于
Summary: We have alot of code duplication whenever we call FullMerge we keep duplicating the instrumentation and statistics code This is a simple diff to refactor the code to use TimedFullMerge instead of FullMerge Test Plan: COMPILE_WITH_ASAN=1 make check -j64 Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59577
-
由 sdong 提交于
Remove the comments saying allow_concurrent_memtable_write and enable_write_thread_adaptive_yield are not stable Summary: allow_concurrent_memtable_write and enable_write_thread_adaptive_yield went through extensive stress test and benchmarks. Should be considered stable. Test Plan: N/A Reviewers: ngbronson, yhchiang, IslamAbdelRahman, dhruba Reviewed By: dhruba Subscribers: MarkCallaghan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59415
-
由 krad 提交于
Summary: The tsan error was because the random implementation we have is not thread safe, using Random::GetTLSInstance Test Plan: Run tests in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59559
-
由 Yi Wu 提交于
Summary: Add option to not flush memtable on open() In case the option is enabled, don't delete existing log files by not updating log numbers to MANIFEST. Will still flush if we need to (e.g. memtable full in the middle). In that case we also flush final memtable. If wal_recovery_mode = kPointInTimeRecovery, do not halt immediately after encounter corruption. Instead, check if seq id of next log file is last_log_sequence + 1. In that case we continue recovery. Test Plan: See unit test. Reviewers: dhruba, horuff, sdong Reviewed By: sdong Subscribers: benj, yhchiang, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57813
-
- 12 6月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: current libgcc lib path is broken, update it and make sure the new one exists Test Plan: verify that the new path exists make check -j64 Reviewers: yiwu, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59535
-
- 11 6月, 2016 10 次提交
-
-
由 Nadav Rotem 提交于
Summary: Try to decompress compressed blocks when a special flag is set. assert and crash in debug builds if we can't decompress the just-compressed input. Test Plan: Run unit-tests. Reviewers: dhruba, andrewkr, sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59145
-
由 sdong 提交于
Summary: Need to change several more places for the change to fix Java tests Test Plan: make jtest under java, run "make db_bench" Reviewers: yhchiang, andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59541
-
由 sdong 提交于
Summary: It confuses some compilers to have slice.cc under multiple directories. Merge them. Test Plan: Run existing tests Reviewers: andrewkr, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59409
-
由 sdong 提交于
Summary: Currently, if users define both of full key bloom and prefix bloom in SST files. During Get(), if full key bloom shows the key may exist, we still go ahead and check prefix bloom. This is wasteful. If bloom filter for full keys exists, we should always ignore prefix bloom in Get(). Test Plan: Run existing tests Reviewers: yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57825
-
由 omegaga 提交于
Summary: delete_scheduler_test and db_sst_test share a same directory name, causing possible fails on both tests when running in parallel. Fixed by changing directory name. Test Plan: Run the two tests in parallel: `parallel -u ./{} ::: delete_scheduler_test db_sst_test` Reviewers: sdong, andrewkr Reviewed By: sdong, andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59529
-
由 Islam AbdelRahman 提交于
Summary: current clang_analyze path is broken, update it Test Plan: USE_CLANG=1 TEST_TMPDIR=/dev/shm/rocksdb OPT=-g make J=64 analyze -j64 make sure it works Reviewers: yiwu, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59505
-
由 Andrew Kryczka 提交于
Summary: The sandcastle setup doesn't provide a remote with our branches. Need to fetch them directly from github. Test Plan: ran script on devserver and the relevant commands on a sandcastle host. Reviewers: IslamAbdelRahman, lightmark, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59511
-
由 sdong 提交于
memtable_prefix_bloom_bits -> memtable_prefix_bloom_bits_ratio and deprecate memtable_prefix_bloom_probes Summary: memtable_prefix_bloom_probes is not a critical option. Remove it to reduce number of options. It's easier for users to make mistakes with memtable_prefix_bloom_bits, turn it to memtable_prefix_bloom_bits_ratio Test Plan: Run all existing tests Reviewers: yhchiang, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: gunnarku, yoshinorim, MarkCallaghan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59199
-
由 krad 提交于
-
由 krad 提交于
Summary: Added new features to help run the entire suite of tests on a diff locally leveraging parallelism Test Plan: Basic command line test Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59445
-
- 10 6月, 2016 8 次提交
-
-
由 Wanning Jiang 提交于
Summary: Backup options file to private directory Test Plan: backupable_db_test.cc, BackupOptions Modify DB options by calling OpenDB for 3 times. Check the latest options file is in the right place. Also check no redundent files are backuped. Reviewers: andrewkr Reviewed By: andrewkr Subscribers: leveldb, dhruba, andrewkr Differential Revision: https://reviews.facebook.net/D59373
-
由 Andrew Kryczka 提交于
Summary: Added an option, --env_uri. When provided, it is used as an argument to NewEnvFromUri(), which instantiates an Env based on it. Test Plan: built a simple binary that registers ChrootEnv for prefix "/", then ran: $ ./tmp --env_uri /tmp/ --db /abcde /tmp/ is the chroot directory and /abcde is the db_name. Then I verified db_bench uses /tmp/abcde Reviewers: sdong, kradhakrishnan, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59325
-
由 Aaron Gao 提交于
Summary: duplicated line in history.md Test Plan: n/a Reviewers: yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59433
-
由 Aaron Gao 提交于
Summary: update version and history Test Plan: N/A Reviewers: yiwu, sdong, IslamAbdelRahman, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59421
-
由 sdong 提交于
Summary: AFIK, options builder is not used by anyone. Remove it. Test Plan: Run all existing tests. Reviewers: IslamAbdelRahman, andrewkr, igor Reviewed By: igor Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59319
-
由 krad 提交于
Summary: Enabled build in Windows and corresponding fixes Test Plan: Compile and run persistent_cache_test in Windows and make check in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59307
-
由 Islam AbdelRahman 提交于
Summary: PersistentCacheOptions class in persistent_cache_tier.h is not used any where yet in the code base but it break the unity build because it have the same name as PersistentCacheOptions in table/persistent_cache_helper.h Remove it temporarily, and the @krad can add it again with a different name when we start using it Test Plan: make unity_test -j64 make check -j64 Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59349
-
由 sdong 提交于
Test Plan: Run the compatibility test Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59385
-
- 09 6月, 2016 4 次提交
-
-
由 dhruba borthakur 提交于
-
由 Andrew Kryczka 提交于
Summary: We had to go back and update the g++ path for 4.4.fb-4.8.fb. So the path is now fixed on the branches, but can't be fixed on the tags since they're immutable. By making format compatibility tests use branch names (when available), backported fixes like this will be used without having to re-release. Also removed v1.5.7 and v2.1 because make fails. Test Plan: $ build_tools/rocksdb-lego-determinator run_format_compatible Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59355
-
由 Islam AbdelRahman 提交于
Summary: Right now we use valgrind built with gcc-5-glibc-2.23 which keep generating this error everywhere ``` ==3224756== Conditional jump or move depends on uninitialised value(s) ==3224756== at 0x401AD68: index (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x4008933: _dl_map_object (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x4000ECC: map_doit (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x40101C7: _dl_catch_error (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x4000B34: do_preload (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x400458C: dl_main (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x40185DB: _dl_sysdep_start (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x4004F39: _dl_start (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== by 0x4000CE7: ??? (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== Uninitialised value was created by a stack allocation ==3224756== at 0x4004519: dl_main (in /usr/local/fbcode/gcc-4.9-glibc-2.20/lib/ld-2.20.so) ==3224756== ``` Test Plan: DISABLE_JEMALLOC=1 make valgrind_check -j64 Reviewers: andrewkr, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59343
-
由 Islam AbdelRahman 提交于
Summary: Disable the test under TSAN temporary to temporarily the build Test Plan: run the test under TSAN Reviewers: kradhakrishnan, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59337
-