- 09 7月, 2016 3 次提交
-
-
由 Yi Wu 提交于
Summary: Fixing erros reported by clang static analyzer. * Removing some unused variables. * Adding assertions to fix false positives reported by clang analyzer. * Adding `__clang_analyzer__` macro to suppress false positive warnings. Test Plan: USE_CLANG=1 OPT=-g make analyze -j64 Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60549
-
由 Gunnar Kudrjavets 提交于
Summary: To make sure that we'll have additional verification for release builds, define a new category and add `make release` to per-diff/post-commit tests. This should in theory prevent the release MyRocks integration builds breaks from happening. Test Plan: - `[p]arc diff --preview` - Observe the execution in Sandcastle and make sure that release build and tests are executed. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60441
-
由 sdong 提交于
Summary: In concurrent memtable insert case, updating counters in MemTable::Add() can count for 5% CPU usage. By batch all the counters and update in the end of the write batch, the CPU overheads are overhead in the use cases where more than one key is updated in one write batch. Test Plan: Write throughput increases 12% with this benchmark setting: TEST_TMPDIR=/dev/shm/ ./db_bench --benchmarks=fillrandom -disable_auto_compactions -level0_slowdown_writes_trigger=9999 -level0_stop_writes_trigger=9999 -num=10000000 --writes=1000000 -max_background_flushes=16 -max_write_buffer_number=16 --threads=64 --batch_size=128 -allow_concurrent_memtable_write -enable_write_thread_adaptive_yield Reviewers: andrewkr, IslamAbdelRahman, ngbronson, igor Reviewed By: ngbronson Subscribers: ngbronson, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60495
-
- 08 7月, 2016 4 次提交
-
-
由 Wanning Jiang 提交于
Summary: this test support CLI option to select HdfsEnv/NativeHdfsEnv now. The latter one is default. add test about when Rename(src, target) should overwrite target Test Plan: existing test Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60399
-
由 Aaron Gao 提交于
Summary: use of dynamic_cast<TransactionImpl*> is unnecessary and also introduce difficulty for fbrocksdb support of TransactionDB Test Plan: ./transaction_test Reviewers: sdong, IslamAbdelRahman, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60501
-
由 Andrew Kryczka 提交于
Summary: Use @omegaga's awesome feature to avoid use of callbacks for ensuring SyncPoints happen in a particular thread. Depends on D60375. Test Plan: $ ./auto_roll_logger_test Reviewers: omegaga, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, omegaga, leveldb Differential Revision: https://reviews.facebook.net/D60471
-
由 omegaga 提交于
Summary: Add markers to sync points. A marked sync point will only be active when it is on the same thread as the marker sync point. Test Plan: Write a unit test to validate. Reviewers: sdong, IslamAbdelRahman, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60375
-
- 07 7月, 2016 4 次提交
-
-
由 Gunnar Kudrjavets 提交于
Summary: MyRocks release integration build breaks because we treat warnings caused by unused variables as errors. Variable `edit` is only used in debug builds. Therefore we need to guard it using `#ifndef NDEBUG` check. Test Plan: - `[p]arc diff --preview` for the default validation. - Verify that release build fails before this fix and passes after applying it. Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60423
-
由 sdong 提交于
Summary: We saw instances where total_log_size is off the real value, but I'm not able to reproduce it. Add more logging to help debugging when it happens again. Test Plan: Run the unit test and see the logging. Reviewers: andrewkr, yhchiang, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60081
-
由 sdong 提交于
Test Plan: Not needed. Reviewers: igor, lgalanis, andrewkr Reviewed By: andrewkr Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60417
-
由 Zun Wang 提交于
Summary: #title Test Plan: n/a Reviewers: uddipta, ldemailly, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60393
-
- 06 7月, 2016 3 次提交
-
-
由 sdong 提交于
Summary: Add option write_buffer_manager to help users control total memory spent on memtables across multiple DB instances. Test Plan: Add a new unit test. Reviewers: yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: adela, benj, sumeet, muthu, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59925
-
由 Aaron Gao 提交于
Summary: Currently, if several flush outputs are committed together, we issue each manifest write per batch (1 batch = 1 flush = 1 sst file = 1+ continuous memtables). Each manifest write requires one fsync and one fsync to parent directory. In some cases, it becomes the bottleneck of write. We should batch them and write in one manifest write when possible. Test Plan: ` ./db_bench -benchmarks="fillseq" -max_write_buffer_number=16 -max_background_flushes=16 -disable_auto_compactions=true -min_write_buffer_number_to_merge=1 -write_buffer_size=65536 -level0_stop_writes_trigger=10000 -level0_slowdown_writes_trigger=10000` **Before** ``` Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 4.9 Date: Fri Jul 1 15:38:17 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 166.277 micros/op 6014 ops/sec; 0.7 MB/s ``` **After** ``` Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 4.9 Date: Fri Jul 1 15:35:05 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 52.328 micros/op 19110 ops/sec; 2.1 MB/s ``` Reviewers: andrewkr, IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: igor, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60075
-
由 omegaga 提交于
Summary: Reported in T11889874. When registering the cleanup function we should copy the option so that we can still access it if ReadOptions is deleted. Test Plan: Add a unit test to reproduce this bug. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60087
-
- 05 7月, 2016 2 次提交
-
-
由 Lucas Qiu 提交于
-
由 Andrew Kryczka 提交于
-
- 02 7月, 2016 3 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: We saw instances where total_log_size is off the real value, but I'm not able to reproduce it. Add more logging to help debugging when it happens again. Test Plan: Run the unit test and see the logging. Reviewers: andrewkr, yhchiang, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60081
-
由 Wanning Jiang 提交于
Summary: LockFile is unnecessary in unit test Test Plan: env_basic_test.cc Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60285
-
由 Wanning Jiang 提交于
Summary: Read only backup engine return NotFound() on missing meta dir (for e2e test) Test Plan: backupable_db_test Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60273
-
- 01 7月, 2016 4 次提交
-
-
由 Aaron Gao 提交于
Summary: The current implementation of rate limiter has the possibility to introduce resource starvation when change its limit. This diff aims to fix this problem by consuming request bytes partially. Test Plan: ``` ./rate_limiter_test [==========] Running 4 tests from 1 test case. [----------] Global test environment set-up. [----------] 4 tests from RateLimiterTest [ RUN ] RateLimiterTest.OverflowRate [ OK ] RateLimiterTest.OverflowRate (0 ms) [ RUN ] RateLimiterTest.StartStop [ OK ] RateLimiterTest.StartStop (0 ms) [ RUN ] RateLimiterTest.Rate request size [1 - 1023], limit 10 KB/sec, actual rate: 10.355712 KB/sec, elapsed 2.00 seconds request size [1 - 1023], limit 20 KB/sec, actual rate: 19.136564 KB/sec, elapsed 2.00 seconds request size [1 - 2047], limit 20 KB/sec, actual rate: 20.783976 KB/sec, elapsed 2.10 seconds request size [1 - 2047], limit 40 KB/sec, actual rate: 39.308144 KB/sec, elapsed 2.10 seconds request size [1 - 4095], limit 40 KB/sec, actual rate: 40.318349 KB/sec, elapsed 2.20 seconds request size [1 - 4095], limit 80 KB/sec, actual rate: 79.667396 KB/sec, elapsed 2.20 seconds request size [1 - 8191], limit 80 KB/sec, actual rate: 81.807158 KB/sec, elapsed 2.30 seconds request size [1 - 8191], limit 160 KB/sec, actual rate: 160.659761 KB/sec, elapsed 2.20 seconds request size [1 - 16383], limit 160 KB/sec, actual rate: 160.700990 KB/sec, elapsed 3.00 seconds request size [1 - 16383], limit 320 KB/sec, actual rate: 317.639481 KB/sec, elapsed 2.50 seconds [ OK ] RateLimiterTest.Rate (22618 ms) [ RUN ] RateLimiterTest.LimitChangeTest [COMPLETE] request size 10 KB, new limit 20KB/sec, refill period 1000 ms [COMPLETE] request size 10 KB, new limit 5KB/sec, refill period 1000 ms [COMPLETE] request size 20 KB, new limit 40KB/sec, refill period 1000 ms [COMPLETE] request size 20 KB, new limit 10KB/sec, refill period 1000 ms [COMPLETE] request size 40 KB, new limit 80KB/sec, refill period 1000 ms [COMPLETE] request size 40 KB, new limit 20KB/sec, refill period 1000 ms [COMPLETE] request size 80 KB, new limit 160KB/sec, refill period 1000 ms [COMPLETE] request size 80 KB, new limit 40KB/sec, refill period 1000 ms [COMPLETE] request size 160 KB, new limit 320KB/sec, refill period 1000 ms [COMPLETE] request size 160 KB, new limit 80KB/sec, refill period 1000 ms [ OK ] RateLimiterTest.LimitChangeTest (5002 ms) [----------] 4 tests from RateLimiterTest (27620 ms total) [----------] Global test environment tear-down [==========] 4 tests from 1 test case ran. (27621 ms total) [ PASSED ] 4 tests. ``` Reviewers: sdong, IslamAbdelRahman, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60207
-
由 Andrew Kryczka 提交于
Summary: Previously we couldn't run env_basic_test on Env::Default (PosixEnv on our platforms) since GetChildren*() behavior was inconsistent with our other Envs. We can normalize the output of GetChildren*() such that these test cases work on PosixEnv too. Test Plan: ran env_basic_test Reviewers: wanning Reviewed By: wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59943
-
由 Andrew Kryczka 提交于
Summary: move cleanup to TearDown and handle directories, so cleanup will happen even if a test fails in the middle. Test Plan: ./env_basic_test Reviewers: wanning Reviewed By: wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60243
-
由 Islam AbdelRahman 提交于
Summary: Travis clang issue on linux is fixed now, re-enable linux on travis Test Plan: https://travis-ci.org/facebook/rocksdb/builds/141098948 Reviewers: yiwu, sdong, yhchiang, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60189
-
- 30 6月, 2016 4 次提交
-
-
由 John Alexander 提交于
* Fixed Windows build error in CMakeLists.txt and perf_level error in db_bench_tool.cc * Changed hard-coded perf levels in db_bench_tool.cc to enum values from perf_level.h * Replaced remaining FLAGS_perf_level > 0
-
由 Wanning Jiang 提交于
Summary: during backup, writable file should call close() before reset() Test Plan: backupable_db_test.cc Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60195
-
由 dhruba borthakur 提交于
-
由 Gunnar Kudrjavets 提交于
Summary: UBSan is unhappy because `cfd` is not initialized. This breaks UBSan build which in turn breaks MyRocks continuous integration with RocksDB which in turns makes me unhappy :-) Fix this. Test Plan: - `[p]arc diff --preview` + Sandcastle. - Verify that `COMPILE_WITH_UBSAN=1 OPT=-g make J=1 ubsan_check` gets past the break. Reviewers: andrewkr, hermanlee4, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60117
-
- 29 6月, 2016 2 次提交
-
-
由 Willem Jan Withagen 提交于
Submitted-by: NWillem Jan Withagen <wjw@digiware.nl>
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff makes the following improvement in regression_test.sh: 1. Add NUM_OPS and DELETE_TEST_PATH to regression_test.sh: * NUM_OPS: The number of operations that will be issued in EACH thread. Default: $NUM_KEYS / $NUM_THREADS * DELETE_TEST_PATH: If true, then the test directory will be deleted after the script ends. Default: 0 2. Add more information in SUMMARY.csv 3. Fix a bug in regression_test.sh where each thread in fillseq will all issue $NUM_KEYS writes. 4. Add --deletes in db_bench, which allows us to control the number of deletes instead of must using FLAGS_num. Test Plan: run regression test with and without DELETE_TEST_PATH and NUM_OPS Reviewers: yiwu, sdong, IslamAbdelRahman, gunnarku Reviewed By: gunnarku Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60039
-
- 28 6月, 2016 2 次提交
-
-
由 sdong 提交于
Update DBTestUniversalCompaction.UniversalCompactionSingleSortedRun to use max_size_amplification_percent = 0 Summary: With max_size_amplification_percent = 0 to make sure that DBTestUniversalCompaction.UniversalCompactionSingleSortedRun tests the configuration to compact to one single sorted run. Test Plan: Run all existing tests Reviewers: yhchiang, andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D60021
-
由 Reid Horuff 提交于
-
- 25 6月, 2016 5 次提交
-
-
由 charsyam 提交于
-
由 Andrew Kryczka 提交于
Summary: Overload RepairDB to take vector-of-ColumnFamilyDescriptor, which tells us CF name + options. Also takes a ColumnFamilyOptions for unspecified column families encountered during the repair. One potentially confusing thing is that we store options in the constructor and don't invoke AddColumnFamily() until discovering the CF in ScanTable. This is because we don't know the CF ID until we find a table belonging to that CF. Depends on D59781. Test Plan: $ ./repair_test Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59853
-
由 Andrew Kryczka 提交于
Summary: This diff uses the CF ID and CF name properties in the SST file to associate recovered data with the proper column family. Depends on D59775. - In ScanTable(), create column families in VersionSet each time a new one is discovered (via reading SST file properties) - In ConvertLogToTable(), dump an SST file for every column family with data in the WAL - In AddTables(), make a VersionEdit per-column family that adds all of that CF's tables Test Plan: $ ./repair_test Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59781
-
由 Wanning Jiang 提交于
Summary: ensure no 2nd level children under test_dir_ Test Plan: env_basic_test on 4 envs Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59979
-
由 Andrew Kryczka 提交于
Summary: To support column families, it is easiest to use VersionSet to manage our column families (if we don't have Versions then ColumnFamilyData always behaves as a dummy column family). This diff only refactors the existing repair logic to use VersionSet; the next two parts will add support for multiple column families. Test Plan: $ ./repair_test Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59775
-
- 24 6月, 2016 3 次提交
-
-
由 Gunnar Kudrjavets 提交于
-
由 Gunnar Kudrjavets 提交于
-
由 Gunnar Kudrjavets 提交于
-
- 23 6月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Fix max_bytes_for_level_base comment Test Plan: none Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59955
-