- 07 3月, 2015 4 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add --thread_status_per_interval to db_bench, which allows db_bench to optionally enable print the current thread status periodically. Test Plan: ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 --max_background_compactions=10 --max_background_flushes=3 --thread_status_per_interval=1000 --key_size=16 --value_size=1000 --num_column_families=10 Sample output: ThreadID ThreadType dbName cfName Operation State 140281571770432 Low Pri 140281575964736 High Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000001 Flush 140281710182464 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000008 Compaction 140281638879296 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000007 Compaction 140281592741952 Low Pri 140281580159040 High Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000002 Flush 140281676628032 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000006 Compaction 140281584353344 Low Pri 140281622102080 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000009 Compaction 140281605324864 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000004 Compaction 140281601130560 High Pri /tmp/rocksdbtest-5297/dbbench default Flush 140281596936256 Low Pri 140281588547648 Low Pri Reviewers: igor, rven, sdong Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34515
-
由 Jim Meyering 提交于
Summary: * src.mk (LIB_SOURCES): Add this file: utilities/document/json_document_builder.cc to avoid link errors. Blame Rev: D33849 Test Plan: run "make" Reviewers: ljin, igor.sugak, rven, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34593
-
由 Jim Meyering 提交于
Summary: Any time one would modify a dependent of any *test*.cc file, "make" would fail to rebuild the affected test binaries, e.g., db_test. That was due to the fact that we deliberately excluded those test-related files from the definition of SOURCES and only $(SOURCES) was used to create the automatically-generated .d dependency files. The fix is to generate a .d file for every source file. * src.mk: New file. Defines LIB_SOURCES, MOCK_SOURCES and TEST_BENCH_SOURCES. * Makefile: Include src.mk. Reflect s/SOURCES/LIB_SOURCES/ renaming. * build_tools/build_detect_platform: Remove the code that was used to generate SOURCES= and MOCK_SOURCES= definitions in make_config.mk. Those lists of files are now hard-coded in src.mk. Hard-coding this list of sources is desirable, because without that, one risks including stray .cc files in a build. Not reproducible. Test Plan: Touch a file used by db_test's dependent .o files and ensure that they are all recompiled. Before, none would be: $ touch db/db_impl.h && make db_test CC db/db_test.o CC db/column_family.o CC db/db_filesnapshot.o CC db/db_impl.o CC db/db_impl_debug.o CC db/db_impl_readonly.o CC db/forward_iterator.o CC db/internal_stats.o CC db/managed_iterator.o CC db/repair.o CC db/write_batch.o CC utilities/compacted_db/compacted_db_impl.o CC utilities/ttl/db_ttl_impl.o CC util/ldb_cmd.o CC util/ldb_tool.o CC util/sst_dump_tool.o CC util/xfunc.o CCLD db_test Reviewers: ljin, igor.sugak, igor, rven, sdong Reviewed By: sdong Subscribers: yhchiang, adamretter, fyrz, dhruba Differential Revision: https://reviews.facebook.net/D33849
-
由 Tian Xia 提交于
Summary: The problem appears to be caused by a bug in Mac OS X compiler (http://llvm.org/bugs/show_bug.cgi?id=15337). We need explicitly construct the base object std::ostream(std::streambuf*) with nullptr. Otherwise, ostream will try to delete the underlying streambuf* which apparently is undefined in the Mac OS X compiler. https://github.com/facebook/rocksdb/issues/525 Test Plan: unit test in fbson make all check document_db_test (on mac) Reviewers: IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34587
-
- 04 3月, 2015 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix a bug in stall time counter. Improve its output format. Test Plan: export ROCKSDB_TESTS=Timeout ./db_test ./db_bench --benchmarks=fillrandom --stats_interval=10000 --statistics=true --stats_per_interval=1 --num=1000000 --threads=4 --level0_stop_writes_trigger=3 --level0_slowdown_writes_trigger=2 sample output: Uptime(secs): 35.8 total, 0.0 interval Cumulative writes: 359590 writes, 359589 keys, 183047 batches, 2.0 writes per batch, 0.04 GB user ingest, stall seconds: 1786.008 ms Cumulative WAL: 359591 writes, 183046 syncs, 1.96 writes per sync, 0.04 GB written Interval writes: 253 writes, 253 keys, 128 batches, 2.0 writes per batch, 0.0 MB user ingest, stall time: 0 us Interval WAL: 253 writes, 128 syncs, 1.96 writes per sync, 0.00 MB written Reviewers: MarkCallaghan, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34275
-
由 Igor Canadi 提交于
Summary: For some reason, libstdc++ implements steady_clock::now() using syscall instead of VDSO optimized clock_gettime() when using glibc 2.16 and earlier. This leads to significant performance degradation for users with older glibcs. See bug reported here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59177 We observed this behavior when testing mongo on AWS hosts. Facebook hosts are unaffected since we use glibc2.17 and 2.20. Revert "Fix timing" This reverts commit 965d9d50. Revert "Use chrono for timing" This reverts commit 001ce64d. Test Plan: make check Reviewers: MarkCallaghan, yhchiang, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34371
-
- 03 3月, 2015 3 次提交
-
-
由 Igor Canadi 提交于
options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. Summary: When having fixed max_bytes_for_level_base, the ratio of size of largest level and the second one can range from 0 to the multiplier. This makes LSM tree frequently irregular and unpredictable. It can also cause poor space amplification in some cases. In this improvement (proposed by Igor Kabiljo), we introduce a parameter option.level_compaction_use_dynamic_max_bytes. When turning it on, RocksDB is free to pick a level base in the range of (options.max_bytes_for_level_base/options.max_bytes_for_level_multiplier, options.max_bytes_for_level_base] so that real level ratios are close to options.max_bytes_for_level_multiplier. Test Plan: New unit tests and pass tests suites including valgrind. Reviewers: MarkCallaghan, rven, yhchiang, igor, ikabiljo Reviewed By: ikabiljo Subscribers: yoshinorim, ikabiljo, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31437
-
由 krad 提交于
Summary: Pre-fetching is a common operation performed by data stores for disk/flash based systems as part of database startup. This is part of task 5197184. Test Plan: Run the newly added unit test Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33933
-
由 Mark Callaghan 提交于
Summary: fix typo Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34251
-
- 28 2月, 2015 5 次提交
-
-
由 Igor Canadi 提交于
Summary: As title Test Plan: compiles Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34191
-
由 Igor Canadi 提交于
Summary: I was pretty sure I compiled this before landing, sorry :/ Test Plan: compiles Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34173
-
由 Igor Canadi 提交于
Summary: This diff fixes a bug introduced by D28521. Read-only backup engine can delete a backup that is later than the latest -- we never check the condition. I also added a bunch of logging that will help with debugging cases like this in the future. See more discussion at t6218248. Test Plan: Added a unit test that was failing before the change. Also, see new LOG file contents: https://phabricator.fb.com/P19738984 Reviewers: benj, sanketh, sumeet, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33897
-
由 Islam AbdelRahman 提交于
Summary: adding stdlib to fbson to fix FreeBSD compilation Test Plan: https://github.com/facebook/mysql-5.6/commit/4cfccf14f4abc5a142b25c54ce1d0f4dafe95f9c Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34143
-
由 Igor Canadi 提交于
-
- 27 2月, 2015 11 次提交
-
-
由 Igor Canadi 提交于
Summary: as title, we have unused variables. this is a short-term solution Test Plan: compiles Reviewers: IslamAbdelRahman, sdong, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34125
-
由 Sameet Agarwal 提交于
Summary: Summary: Added a new option to ColumnFamllyOptions - optimize_filters_for_hits. This option can be used in the case where most accesses to the store are key hits and we dont need to optimize performance for key misses. This is useful when you have a very large database and most of your lookups succeed. The option allows the store to not store and use filters in the last level (the largest level which contains data). These filters can take a large amount of space for large databases (in memory and on-disk). For the last level, these filters are only useful for key misses and not for key hits. If we are not optimizing for key misses, we can choose to not store these filters for that level. This option is only provided for BlockBasedTable. We skip the filters when we are compacting Test Plan: 1. Modified db_test toalso run tests with an additonal option (skip_filters_on_last_level) 2. Added another unit test to db_test which specifically tests that filters are being skipped Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: lgalanis, yoshinorim, MarkCallaghan, rven, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33717
-
由 Islam AbdelRahman 提交于
Summary: This patch will update the Makefile and source code so that we can build RocksDB successfully on FreeBSD 10 and 11 (64-bit and 32-bit) I have also encountered some problems when running tests on FreeBSD, I will try to fix them individually in different diffs Notes: - FreeBSD uses clang as it's default compiler (http://lists.freebsd.org/pipermail/freebsd-current/2012-September/036480.html) - GNU C++ compiler have C++ 11 problems on FreeBSD (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193528) - make is not gmake on FreeBSD (http://www.khmere.com/freebsd_book/html/ch01.html) Test Plan: Using VMWare Fusion Create 4 VM machines (FreeBSD 11 64-bit, FreeBSD 11 32-bit, FreeBSD 10 64-bit, FreeBSD 10 32-bit) - pkg install git gmake gflags archivers/snappy - git clone https://github.com/facebook/rocksdb.git - apply this patch - setenv CXX c++ - setenv CPATH /usr/local/include/ - setenv LIBRARY_PATH /usr/local/lib/ - gmake db_bench - make sure compilation is successful and db_bench is running - gmake all - make sure compilation is successful Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33891
-
由 Igor Sugak 提交于
Summary: Current clang path in fbcode points to clang 3.5. This diff updates clang path to clang 3.7. Test Plan: Make sure clang 3.7 is in use and all tests are passing. ```lang=bash % USE_CLANG make check ``` Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34119
-
由 Venkatesh Radhakrishnan 提交于
Summary: The LoadDependency function does not take a lock when it runs and it could be modifying data structures while other threads are accessing it. Test Plan: Run TSAN. Reviewers: igor, sdong Reviewed By: igor, sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34095
-
由 Igor Sugak 提交于
Summary: When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Some errors are uninitialized use errors. ``` ... CC db/log_test.o util/ldb_cmd.cc:394:16: error: base class 'rocksdb::LDBCommand' is uninitialized when used here to access 'rocksdb::LDBCommand::BuildCmdLineOptions' [-Werror,-Wuninitialized] BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX, ^ ... ``` ```lang=c++ CompactorCommand::CompactorCommand(const vector<string>& params, const map<string, string>& options, const vector<string>& flags) : LDBCommand(options, flags, false, BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX, ARG_TTL})), null_from_(true), null_to_(true) { . . . } ``` For the fourth parameter of the base constructor (`LDBCommand`) we call `BuildCmdLineOptions`, which is a private non-static method of `LDBCommand` base class. This diff adds missing `static` keyword for `LDBCommand::BuildCmdLineOptions` method. Test Plan: Build with trunk clang and make sure all tests are passing. ```lang=bash % # Have trunk clang present in path. % ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make check `` Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34083
-
由 Adam Retter 提交于
[RocksJava] RocksJava Testcases
-
由 fyrz 提交于
- NativeLibraryLoader Test - EnvironmentTest Bugfix
-
由 stash93 提交于
Summary: mac compile is fixed in fbson, so it can be returned back from 7ce1b2c1 Test Plan: make all check make valgrind_check Reviewers: golovachalexander, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33855
-
由 Igor Sugak 提交于
Summary: When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Almost all of them are missing override errors. This diff adds missing override keyword. No manual changes. Prerequisites: bear and clang 3.5 build with extra tools ```lang=bash % USE_CLANG=1 bear make all # generate a compilation database http://clang.llvm.org/docs/JSONCompilationDatabase.html % clang-modernize -p . -include . -add-override % make format ``` Test Plan: Make sure all tests are passing. ```lang=bash % #Use default fb code clang. % make check ``` Verify less error and no missing override errors. ```lang=bash % # Have trunk clang present in path. % ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make ``` Reviewers: igor, kradhakrishnan, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34077
-
由 Mark Callaghan 提交于
Summary: This changes the RocksDB build to share headers provided by MongoDB. It is invoked as: ROCKSDB_FOR_MONGO=/path/to/mongodb/root make static_lib Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34071
-
- 26 2月, 2015 7 次提交
-
-
由 Adam Retter 提交于
[RocksJava] Final usage correction
-
由 krad 提交于
Summary: BlockBasedTable pre-fetches the filter and index blocks on Open call. This is an optimistic optimization targeted for runtime scenario. The optimization is unnecessary for sst_dump_tool - Added a provision to disable pre-fetching of index and filter blocks in BlockBasedTable - Disabled pre-fetching for the sst_dump tool Stack for reference : #01 0x00000000005ed944 in snappy::InternalUncompress<snappy::SnappyArrayWriter> () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:148 #02 0x00000000005edeee in snappy::RawUncompress () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:947 #03 0x00000000004e0b4d in rocksdb::UncompressBlockContents () from /data/users/paultuckfield/rocksdb/./util/compression.h:69 #04 0x00000000004e145c in rocksdb::ReadBlockContents () from /data/users/paultuckfield/rocksdb/table/format.cc:334 #05 0x00000000004ca424 in rocksdb::(anonymous namespace)::ReadBlockFromFile () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:70 #06 0x00000000004cccad in rocksdb::BlockBasedTable::CreateIndexReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:173 #07 0x00000000004d17e5 in rocksdb::BlockBasedTable::Open () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:553 #08 0x00000000004c8184 in rocksdb::BlockBasedTableFactory::NewTableReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_factory.cc:51 #09 0x0000000000598463 in rocksdb::SstFileReader::NewTableReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:69 #10 0x00000000005986c2 in rocksdb::SstFileReader::SstFileReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:26 #11 0x0000000000599047 in rocksdb::SSTDumpTool::Run () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:332 #12 0x0000000000409b06 in main () from /data/users/paultuckfield/rocksdb/tools/sst_dump.cc:12 Test Plan: - Added a unit test to trigger the code. - Also did some manual verification. - Passed all unit tests task #6296048 Reviewers: igor, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34041
-
由 Mark Callaghan 提交于
Summary: An old commit (482401) changed DoWrite to use the value of --writes rather than --num to determine the range for keys. This restores the old and correct behavior which is to limit it using --num. Task ID: #6353043 Blame Rev: Test Plan: run db_bench Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34065
-
由 Igor Canadi 提交于
[RocksJava] Updated TickerTypes and Histogram to 3.10
-
由 fyrz 提交于
Updated tickers and histogram types to 3.10.
-
由 fyrz 提交于
-
由 fyrz 提交于
Summary: Introduced final keyword to parameters with immutable values and classes which should not be derived. Test Plan: make rocksdbjava make jtest Reviewers: yhchiang, adamretter, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33417
-
- 25 2月, 2015 8 次提交
-
-
由 Igor Canadi 提交于
-
由 Venkatesh Radhakrishnan 提交于
Summary: In a release build, a member was not being accessed. This member was only being accessed in a debug build. We now add an accessor function for this member and the buid succeeds. Test Plan: build release/unity/debug on linux/mac Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34035
-
由 Igor Canadi 提交于
This reverts commit bd339a97. Conflicts: db/managed_iterator.cc
-
由 Igor Canadi 提交于
-
由 Adam Retter 提交于
[RocksJava] Raw use of parametrized class
-
由 Adam Retter 提交于
[RocksJava] Redundant access-modifier in interfaces
-
由 Igor Canadi 提交于
Summary: See https://www.facebook.com/groups/rocksdb.dev/permalink/718536351578215/. We should enable people to run without warning-as-error, since we break this so often for different compilers. Test Plan: introduce an warning, compile sucessfully Reviewers: yhchiang, meyering, igor.sugak, rven, sdong Reviewed By: sdong Subscribers: adamretter, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33963
-
由 Igor Canadi 提交于
Summary: This causes warnings on OS X Test Plan: compiles Reviewers: rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33969
-