- 07 6月, 2016 6 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch does the following improvement for regression_test.sh * Allow regression_test.sh to specify OPTIONS_FILE. * Add header comments that includes examples on how to run the script and introduce all configurable parameters. * bug fix. Test Plan: Run the example commands in the header comments of regression_test.sh Reviewers: sdong, yiwu, gunnarku Reviewed By: gunnarku Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59175
-
由 krad 提交于
Summary: Fix 2 issues that was breaking Windows build 1) double to size_t potential downcast warning 2) port_posix is not ready for windows, avoiding building hash_table_bench to avoid build break Test Plan: compile in Windoes and make check Reviewers: sdong, andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59265
-
由 Yi Wu 提交于
Summary: clang was removed from tp2 so change to llvm-fb. Test Plan: USE_CLANG=1 make all -j64 Reviewers: yhchiang, andrewkr, kradhakrishnan, sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59259
-
-
由 krad 提交于
Summary: This diff provides the basic interface definitions of persistent read cache system PersistentCacheOptions captures the persistent read cache options used to configure and control the system PersistentCacheTier provides the basic building block for constructing tiered cache PersistentTieredCache provides a logical abstraction of tiers of cache layered over one another Test Plan: Compile Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57051
-
由 Andrew Kryczka 提交于
Summary: The path to g++ was outdated. Ran ./build_tools/update_dependencies.sh. Test Plan: make -j64 all Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59229
-
- 06 6月, 2016 3 次提交
-
-
由 Willem Jan Withagen 提交于
* stack_trace,cc: The current Stacktrace code does not compile for FreeBSD So set it to generate empty routines * stack_trace,cc: The current Stacktrace code does not compile for FreeBSD Use the definition also used in other commits
-
由 Willem Jan Withagen 提交于
-
由 Willem Jan Withagen 提交于
-
- 04 6月, 2016 5 次提交
-
-
由 Andrew Kryczka 提交于
Summary: - Provide env_test as a static library. We will build it for future releases so internal Envs can use env_test by linking against this library. - Add tests for CustomEnv, which is configurable via ENV_TEST_URI environment variable. It uses the URI-based Env lookup (depends on D58449). - Refactor env_basic_test cases to use a unique/configurable directory for test files. Test Plan: built a test binary against librocksdb_env_test.a. It registered the default Env with URI prefix "a://". - verify runs all CustomEnv tests when URI with correct prefix is provided ``` $ ENV_TEST_URI="a://ok" ./tmp --gtest_filter="CustomEnv/*" ... [ PASSED ] 12 tests. ``` - verify runs no CustomEnv tests when URI with non-matching prefix is provided ``` $ ENV_TEST_URI="b://ok" ./tmp --gtest_filter="CustomEnv/*" ... [ PASSED ] 0 tests. ``` Reviewers: ldemailly, IslamAbdelRahman, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58485
-
由 Uddipta Maity 提交于
Summary: Rocksdb backup and restore rate limiting is currently done per backup/restore. So, it is difficult to control rate across multiple backup/restores. With this change, a throttler can be provided. If a throttler is provided, it is used. Otherwise, a new throttler is created based on the actual rate limits specified in the options. Test Plan: Added unit tests Reviewers: ldemailly, andrewkr, sdong Reviewed By: andrewkr Subscribers: igor, yiwu, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56265
-
由 Andrew Kryczka 提交于
Summary: Extracted basic Env-related tests from mock_env_test and memenv_test into a parameterized test for Envs: env_basic_test. Depends on D58449. (The dependency is here only so I can keep this series of diffs in a chain -- there is no dependency on that diff's code.) Test Plan: ran tests Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58635
-
由 Aaron Gao 提交于
Summary: With `table_options.cache_index_and_filter_blocks = true`, index and filter blocks are stored in block cache. Then people are curious how much of the block cache total size is used by indexes and bloom filters. It will be nice we have a way to report that. It can help people tune performance and plan for optimized hardware setting. We add several enum values for db Statistics. BLOCK_CACHE_INDEX/FILTER_BYTES_INSERT - BLOCK_CACHE_INDEX/FILTER_BYTES_ERASE = current INDEX/FILTER total block size in bytes. Test Plan: write a test case called `DBBlockCacheTest.IndexAndFilterBlocksStats`. The result is: ``` [gzh@dev9927.prn1 ~/local/rocksdb] make db_block_cache_test -j64 && ./db_block_cache_test --gtest_filter=DBBlockCacheTest.IndexAndFilterBlocksStats Makefile:101: Warning: Compiling in debug mode. Don't use the resulting binary in production GEN util/build_version.cc make: `db_block_cache_test' is up to date. Note: Google Test filter = DBBlockCacheTest.IndexAndFilterBlocksStats [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBBlockCacheTest [ RUN ] DBBlockCacheTest.IndexAndFilterBlocksStats [ OK ] DBBlockCacheTest.IndexAndFilterBlocksStats (689 ms) [----------] 1 test from DBBlockCacheTest (689 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (689 ms total) [ PASSED ] 1 test. ``` Reviewers: IslamAbdelRahman, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58677
-
由 Adam Retter 提交于
-
- 03 6月, 2016 7 次提交
-
-
由 Andrew Kryczka 提交于
Summary: This enables configurable Envs without recompiling. For example, my next diff will make env_test test an Env created by NewEnvFromUri(). Then, users can determine which Env is tested simply by providing the URI for NewEnvFromUri() (e.g., through a CLI argument or environment variable). The registration process allows us to register any Env that is linked with the RocksDB library, so we can register our internal Envs as well. The registration code is inspired by our internal InitRegistry. Test Plan: new unit test Reviewers: IslamAbdelRahman, lightmark, ldemailly, sdong Reviewed By: sdong Subscribers: leveldb, dhruba, andrewkr Differential Revision: https://reviews.facebook.net/D58449
-
由 Jan Doms 提交于
-
由 Aaron Gao 提交于
Summary: fix flaky test Test Plan: `make all check` Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59157
-
由 krad 提交于
Summary: Add hash table (under persistent cache) to CMake list Test Plan: Run hash_test in windows and make check in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59151
-
由 Andrew Kryczka 提交于
Summary: MemTableList::current_ could be written by background flush thread and simultaneously read in the user thread (NumNotFlushed() is used in SwitchMemtable()). Use the lock to prevent this case. Found the error from tsan. Related: D58833 Test Plan: $ OPT=-g COMPILE_WITH_TSAN=1 make -j64 db_test $ TEST_TMPDIR=/dev/shm/rocksdb ./db_test --gtest_filter=DBTest.RepeatedWritesToSameKey Reviewers: lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59139
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch allows db_bench to initialize it's RocksDB Options via a options file, specified by the --options_file flag. Note that if --options_file flag is set, then it has higher priority than the command-line argument. Test Plan: db_bench_tool_test Reviewers: sdong, IslamAbdelRahman, kradhakrishnan, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58533
-
由 PraveenSinghRao 提交于
* Create a callback for memtable becoming immutable Create a callback for memtable becoming immutable Create a callback for memtable becoming immutable moved notification outside the lock Move sealed notification to unlocked portion of SwitchMemtable * fix lite build
-
- 02 6月, 2016 5 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch does the following improvement on the regression_test.sh * allows db_bench being executed at a remost host while storing the benchmark results locally. * kills all db_bench related processes before running db_bench * better error handling. Test Plan: 1. Run regression_test.sh both locally and remotely 2. Run multiple regression_test.sh at the same time and make sure i. Only one runs successfully. ii. The one that runs successfully will kill all other db_bench processes before it runs any benchmark. Reviewers: sdong, yiwu, gunnarku Reviewed By: gunnarku Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58611
-
由 krad 提交于
Summary: Direct IO checkin breaks Windows build. Fixing the code to work for Windows. Test Plan: Run env_test in Windows 10 and make check in Linux Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59073
-
由 Mike Kolupaev 提交于
Summary: We see some write stalls because of number of unflushed memtables. With existing logging I couldn't figure out what's happening exactly. See internal task t11446054 for details if interested. This diff adds: - logging of memtable creation at info level; I wanted it on multiple occasions for different reasons; also include number of immutable memtables, - logging of number of remaining immutable memtables after a flush. Test Plan: ran tests Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58833
-
由 siddontang 提交于
-
由 Vasile Paraschiv 提交于
Summary: Add asserts around PerfLevel enum Test Plan: make all check -j32 Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D59007
-
- 01 6月, 2016 2 次提交
-
-
由 Reid Horuff 提交于
-
由 sdong 提交于
Summary: The function wrapper for LDBCommand::SelectCommand is too long so that Windows build fails with warning "decorated name length exceeded, name was truncated". Shrink the length by using a struct. Test Plan: Build on both of Linux and Windows and make sure the warning doesn't show in either platform. Reviewers: andrewkr, adsharma, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58965
-
- 30 5月, 2016 1 次提交
-
-
由 Zuoyan Qin 提交于
-
- 28 5月, 2016 6 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Currently all the tools are included in librocksdb.a (db_bench is not). With this separate library, we can access db_bench functionality from our internal repo and eventually move tools out of librocksdb.a. Test Plan: built a simple binary against this library that invokes db_bench_tool(). Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58977
-
由 sdong 提交于
Summary: When rate_bytes_per_sec * refill_period_us_ overflows, the actual limited rate is very low. Handle this case so the rate will be large. Test Plan: Add a unit test for it. Reviewers: IslamAbdelRahman, andrewkr Reviewed By: andrewkr Subscribers: yiwu, lightmark, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58929
-
由 Andrew Kryczka 提交于
Summary: Made it consistent with posix Env, which uses pread() that returns 0 (success) when an offset is given beyond EOF. The purpose of making these Envs behave consistently is I am repurposing the in-memory Envs' tests for the basic Env tests in D58635. Test Plan: ran mock_env_test and memenv_test Reviewers: IslamAbdelRahman, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58845
-
由 sdong 提交于
Summary: Build failure with some compiler setting with tools/reduce_levels_test.cc:97: undefined reference to `rocksdb::LDBCommand* rocksdb::LDBCommand::InitFromCmdLineArgs<rocksdb::LDBCommand* (*)(std::string const&, std::vector<std::string, std::allocator<std::string> > const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&)>(std::vector<std::string, std::allocator<std::string> > const&, rocksdb::Options const&, rocksdb::LDBOptions const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const*, rocksdb::LDBCommand* (*)(std::string const&, std::vector<std::string, std::allocator<std::string> > const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&))' Fix it by changing to function pointer instead Test Plan: Run all existing tests Reviewers: andrewkr, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: adsharma, lightmark, yiwu, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58905
-
由 Vasile Paraschiv 提交于
Summary: Add under and over limits for rocksdb::PerfLevel enum to allow us to do boundary checks before casting ints or unints to this enum. Test Plan: make all check -j32 Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58521
-
由 sdong 提交于
Summary: util/threadpool.cc's function name is the same as a well-known class name. It breaks unity build. Rename it. Test Plan: Run all existing test. Reviewers: yiwu, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58881
-
- 27 5月, 2016 2 次提交
-
-
由 sdong 提交于
Summary: I don't think util/mutable_cf_options.h is needed inside include/rocksdb/sst_file_writer.h and it is not allowed. Remove it. Test Plan: Run all tests Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: lightmark, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58827
-
由 sdong 提交于
Summary: We added more table properties for each SST file, so when using 2KB SST file size, the estimated size of SST files is off by almost half, causing the LSM tree structure not as expected. Fix it by making file size 4x as previously, as well as LSM base size. Also avoid the sleeping based synchronization and turn to use sync points. Test Plan: Run paralell unit tests multiple times and make sure they always pass. Reviewers: IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58749
-
- 25 5月, 2016 2 次提交
-
-
由 krad 提交于
-
由 Islam AbdelRahman 提交于
Summary: Fix simple issue with FLAGS_simcache_size condition Test Plan: run db_bench Reviewers: lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58743
-
- 24 5月, 2016 1 次提交
-
-
由 Aaron Gao 提交于
Summary: add class SimCache(base class with instrumentation api) and SimLRUCache(derived class with detailed implementation) which is used as an instrumented block cache that can predict hit rate for different cache size Test Plan: Add a test case in `db_block_cache_test.cc` called `SimCacheTest` to test basic logic of SimCache. Also add option `-simcache_size` in db_bench. if set with a value other than -1, then the benchmark will use this value as the size of the simulator cache and finally output the simulation result. ``` [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 1000000 RocksDB: version 4.8 Date: Tue May 17 16:56:16 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 6.809 micros/op 146874 ops/sec; 16.2 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.343 micros/op 157665 ops/sec; 17.4 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 986559 SimCache HITs: 264760 SimCache HITRATE: 26.84% [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 10000000 RocksDB: version 4.8 Date: Tue May 17 16:57:10 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 5.066 micros/op 197394 ops/sec; 21.8 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.457 micros/op 154870 ops/sec; 17.1 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 1059764 SimCache HITs: 374501 SimCache HITRATE: 35.34% [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 100000000 RocksDB: version 4.8 Date: Tue May 17 16:57:32 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 5.632 micros/op 177572 ops/sec; 19.6 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.892 micros/op 145094 ops/sec; 16.1 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 1150767 SimCache HITs: 1034535 SimCache HITRATE: 89.90% ``` Reviewers: IslamAbdelRahman, andrewkr, sdong Reviewed By: sdong Subscribers: MarkCallaghan, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57999
-