- 14 3月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Here's my proposal for making our LOGs easier to read by machines. The idea is to dump all events as JSON objects. JSON is easy to read by humans, but more importantly, it's easy to read by machines. That way, we can parse this, load into SQLite/mongo and then query or visualize. I started with table_create and table_delete events, but if everybody agrees, I'll continue by adding more events (flush/compaction/etc etc) Test Plan: Ran db_bench. Observed: 2015/01/15-14:13:25.788019 1105ef000 EVENT_LOG_v1 {"time_micros": 1421360005788015, "event": "table_file_creation", "file_number": 12, "file_size": 1909699} 2015/01/15-14:13:25.956500 110740000 EVENT_LOG_v1 {"time_micros": 1421360005956498, "event": "table_file_deletion", "file_number": 12} Reviewers: yhchiang, rven, dhruba, MarkCallaghan, lgalanis, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31647
-
- 12 3月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fixed a bug where CompactFiles won't delete obsolete files until flush. Test Plan: ./compact_files_test export ROCKSDB_TESTS=CompactFiles ./db_test Reviewers: rven, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34671
-
- 07 3月, 2015 1 次提交
-
-
由 Jim Meyering 提交于
Summary: Any time one would modify a dependent of any *test*.cc file, "make" would fail to rebuild the affected test binaries, e.g., db_test. That was due to the fact that we deliberately excluded those test-related files from the definition of SOURCES and only $(SOURCES) was used to create the automatically-generated .d dependency files. The fix is to generate a .d file for every source file. * src.mk: New file. Defines LIB_SOURCES, MOCK_SOURCES and TEST_BENCH_SOURCES. * Makefile: Include src.mk. Reflect s/SOURCES/LIB_SOURCES/ renaming. * build_tools/build_detect_platform: Remove the code that was used to generate SOURCES= and MOCK_SOURCES= definitions in make_config.mk. Those lists of files are now hard-coded in src.mk. Hard-coding this list of sources is desirable, because without that, one risks including stray .cc files in a build. Not reproducible. Test Plan: Touch a file used by db_test's dependent .o files and ensure that they are all recompiled. Before, none would be: $ touch db/db_impl.h && make db_test CC db/db_test.o CC db/column_family.o CC db/db_filesnapshot.o CC db/db_impl.o CC db/db_impl_debug.o CC db/db_impl_readonly.o CC db/forward_iterator.o CC db/internal_stats.o CC db/managed_iterator.o CC db/repair.o CC db/write_batch.o CC utilities/compacted_db/compacted_db_impl.o CC utilities/ttl/db_ttl_impl.o CC util/ldb_cmd.o CC util/ldb_tool.o CC util/sst_dump_tool.o CC util/xfunc.o CCLD db_test Reviewers: ljin, igor.sugak, igor, rven, sdong Reviewed By: sdong Subscribers: yhchiang, adamretter, fyrz, dhruba Differential Revision: https://reviews.facebook.net/D33849
-
- 28 2月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
-
- 27 2月, 2015 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: This patch will update the Makefile and source code so that we can build RocksDB successfully on FreeBSD 10 and 11 (64-bit and 32-bit) I have also encountered some problems when running tests on FreeBSD, I will try to fix them individually in different diffs Notes: - FreeBSD uses clang as it's default compiler (http://lists.freebsd.org/pipermail/freebsd-current/2012-September/036480.html) - GNU C++ compiler have C++ 11 problems on FreeBSD (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193528) - make is not gmake on FreeBSD (http://www.khmere.com/freebsd_book/html/ch01.html) Test Plan: Using VMWare Fusion Create 4 VM machines (FreeBSD 11 64-bit, FreeBSD 11 32-bit, FreeBSD 10 64-bit, FreeBSD 10 32-bit) - pkg install git gmake gflags archivers/snappy - git clone https://github.com/facebook/rocksdb.git - apply this patch - setenv CXX c++ - setenv CPATH /usr/local/include/ - setenv LIBRARY_PATH /usr/local/lib/ - gmake db_bench - make sure compilation is successful and db_bench is running - gmake all - make sure compilation is successful Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33891
-
- 25 2月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: See https://www.facebook.com/groups/rocksdb.dev/permalink/718536351578215/. We should enable people to run without warning-as-error, since we break this so often for different compilers. Test Plan: introduce an warning, compile sucessfully Reviewers: yhchiang, meyering, igor.sugak, rven, sdong Reviewed By: sdong Subscribers: adamretter, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33963
-
- 24 2月, 2015 1 次提交
-
-
由 Igor Sugak 提交于
Summary: * Updated Makefile to exit `make analyze` with status 1 if scan-build detected any bugs. * scan-build automatically detects which c++ compiler to use, and some times is uses wrong ones (from $CPP). Added implicit parameters to use $CC and $CXX. * Added `scan_build_report` directory to .gitignore file. * Added `scan_build_report` directory to clean target. Test Plan: Run `make analyze` and verify that exit status is 1, if there are scan-build bugs detected. Run `make clean` and verify that files in `scan_build_report` directory are deleted. After running `make analyze; git status` and verify that no untracked files in `scan_build_report` directory. Reviewers: meyering, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33831
-
- 21 2月, 2015 1 次提交
-
-
由 Jim Meyering 提交于
Summary: * Makefile (WARNING_FLAGS): Add -W and -Wextra, and at least for now, -Wno-unused-parameter. Without that latter one, there would be many errors/warnings about unused parameters. Test Plan: run these and ensure there is no warning: make clean && make make clean && USE_CLANG=1 make Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33819
-
- 20 2月, 2015 2 次提交
-
-
由 Jim Meyering 提交于
Summary: Prior to this change, "make check" would always waste a lot of time relinking 60+ binaries. With this change, it does that only when the generated file, util/build_version.cc, changes, and that happens only when the date changes or when the current git SHA changes. This change makes some other improvements: before, there was no rule to build a deleted util/build_version.cc. If it was somehow removed, any attempt to link a program would fail. There is no longer any need for the separate file, build_tools/build_detect_version. Its functionality is now in the Makefile. * Makefile (DEPFILES): Don't filter-out util/build_version.cc. No need, and besides, removing that dependency was wrong. (date, git_sha, gen_build_version): New helper variables. (util/build_version.cc): New rule, to create this file and update it only if it would contain new information. * build_tools/build_detect_platform: Remove file. * db/db_impl.cc: Now, print only date (not the time). * util/build_version.h (rocksdb_build_compile_time): Remove declaration. No longer used. Test Plan: - Run "make check" twice, and note that the second time no linking is performed. - Remove util/build_version.cc and ensure that any "make" command regenerates it before doing anything else. - Run this: strings librocksdb.a|grep _build_. That prints output including the following: rocksdb_build_git_date:2015-02-19 rocksdb_build_git_sha:2.8.fb-1792-g3cb6cc0 Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33591
-
由 Jim Meyering 提交于
Summary: The "ar" command was excessively verbose. This abbreviates it to be like CC and LD. Also, factor "rs" options into ARFLAGS. Test Plan: $ rm librocksdb.a $ make librocksdb.a AR librocksdb.a ar: creating librocksdb.a Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33699
-
- 19 2月, 2015 4 次提交
-
-
由 Jim Meyering 提交于
Summary: Many link commands were identical. Factor that out into a variable, AM_LINK, and use it in place of all of those open-coded commands. Test Plan: run "make check" Reviewers: ljin, igor, sdong Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33585
-
由 Jim Meyering 提交于
Summary: With this change, make now prints a summary line for each compiler and linker invocation, e.g.,: CC db/builder.o CC db/c.o CC db/column_family.o To see full commands, insert "V=1" into your make command. E.g., run "make V=1 all" if you want it to print each command in its full glory. $^ is GNU make's abbreviation for the prerequisites of the current target. These AM_V_... variables expand to some very short string like "CC" or "LD", by default, so that the output of "make" is readable. If/when you want more details, just build with "make V=1 ...", and make will print each full command as it is executed. If you prefer to see the noise all the time, and only want to optionally see the abbreviated output, set AM_DEFAULT_VERBOSITY=1 in your environment, and then build with V=0 to see the abbreviated command indicators. Test Plan: invoke make a few different ways and observe: make clean; make # abbreviated make clean; make V=0 # also abbreviated make clean; make V=1 # full detail Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33579
-
由 Jim Meyering 提交于
Summary: This is in preparation for some factorization. * Makefile (deletefile_test): Add $(COVERAGEFLAGS) to link command. (options_test): Remove explicit (redundant) dependency on options_helper.o: that is already a dependent, via $(LIBOBJECTS) Test Plan: run make Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33573
-
由 Jim Meyering 提交于
Summary: There were Makefile rules to build those two targets, but neither rule has worked for a long time, due to missing dependent source files. Remove those rules. Test Plan: run "make" Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33567
-
- 18 2月, 2015 1 次提交
-
-
由 Jim Meyering 提交于
Summary: Before this change, running "make" with no arguments would silently run the rules for the "uninstall" target(!). Don't do that. * Makefile (default): New, first target; depend on "all". (uninstall, install): Do not hide the commands we run. Test Plan: Run "make" and verify that the rules for "uninstall" are no longer run. Instead, note that many files are compiled and linked. Before, none were. Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33531
-
- 14 2月, 2015 2 次提交
-
-
由 Jim Meyering 提交于
Summary: * Makefile (dummy): Prefix this statement with "dummy := ", so that it no longer triggers a syntax error from GNU make 3.80 and earlier. Reported by nielsl in https://github.com/facebook/rocksdb/issues/509 Test Plan: run make Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33429
-
由 Jim Meyering 提交于
Summary: The code being removed would invoke sed differently to work around a portability difference in how sed -i works (different on MacOS). Yet performing a host-type-based ifdef fails when the tools installed do not match. That sed use was solely to post-process the .d file. Instead, generate the desired output directly, by using the compiler's -MT<FILE> option. * Makefile (%.d: %.cc): With the prior use of Makefile-ifdef'd sed, when building on MacOS with gnu sed, every run of this rule would fail with a sed usage error. Also list each .d file as a dependent. Test Plan: Ensure that a selected .d file is the same as before both with g++ and with clang++. However, note that the new .d files each contain a new reference to the .d file itself. Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33369
-
- 05 2月, 2015 2 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
-
- 04 2月, 2015 1 次提交
-
-
由 Igor Sugak 提交于
Summary: Added new target ##make analyze## into Makefile. This command runs clang static analyzer and builds the sources as ##make all##. The result report is put into ##$(RocksDbSourceRoot)/can_build_report/## If the development environment is a Facebook devserver and ##ROCKSDB_NO_FBCODE## is not set, then scan-build is used from fbcode. If it is run not on a Facebook devserver, scan-build should be available in ##$PATH##. I'll add details to wiki how to install scan-build on a non Facebook devserver environment. Test Plan: Run the fallowing commands on a Facebook devserver and Mac OS, and ensure no build or test errors. ``` % make all check -j32 % make clean % USE_CLANG=1 make all -j32 % make analyze % USE_CLANG=1 make analyze ``` Reviewers: sdong, lgalanis, leveldb, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D32799
-
- 02 2月, 2015 1 次提交
-
-
由 Adam Retter 提交于
-
- 31 1月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: Add CappedFixTransform, which is the same as fixed length prefix extractor, except that when slice is shorter than the fixed length, it will use the full key. Test Plan: Add a test case for db_test options_test and a new test Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: MarkCallaghan, leveldb, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D31887
-
- 30 1月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: This was a feature request by osquery. See task t5617758 Test Plan: compiles and memenv_test runs Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D32115
-
- 29 1月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: There's a bug in TSAN (or libstdc++?) with std::shared_ptr<> for some reason. In db_test, only FlushSchedule is affected. See more: https://groups.google.com/forum/#!topic/thread-sanitizer/vz_s-t226Vg With this change and all other @sdong's and mine diffs, our db_test should be TSAN-clean. I'll move to other tests. Test Plan: no more flush schedule when running TSAN Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D32469
-
- 28 1月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: We don't have plans to work on this in the short term. If we ever resurrect the project, we can find the code in the history. No need for it to linger around Test Plan: no test Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D32349
-
- 27 1月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: 1. If WAL directory is different from db directory. Sync the directory after creating a log file under it. 2. After creating an SST file, sync its parent directory instead of DB directory. 3. change the check of kResetDeleteUnsyncedFiles in fault_injection_test. Since we changed the behavior to sync log files' parent directory after first WAL sync, instead of creating, kResetDeleteUnsyncedFiles will not guarantee to show post sync updates. Test Plan: make all check Reviewers: yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D32067
-
- 24 1月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: When you compile with COMPILE_WITH_TSAN=1, we will compile the code with -fsanitize=thread. This will resolve bunch of data race issues we might have. Test Plan: COMPILE_WITH_TSAN=1 m db_test Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D32019
-
- 16 1月, 2015 1 次提交
-
-
由 Thomas Dudziak 提交于
Summary: This is a port of [[ https://github.com/google/leveldb/blob/master/db/fault_injection_test.cc | LevelDB's fault_injection_test ]] to RocksDB. Unfortunately it fails with: ``` ==== Test FaultInjectionTest.FaultTest db/fault_injection_test.cc:491: Corruption: no meta-nextfile entry in descriptor #0 ./fault_injection_test() [0x41477a] rocksdb::FaultInjectionTest::PartialCompactTestReopenWithFault(rocksdb::FaultInjectionTest::ResetMethod, int, int) /data/users/tomdzk/rocksdb/db/fault_injection_test.cc:491 #1 ./fault_injection_test() [0x40a38a] rocksdb::_Test_FaultTest::_Run() /data/users/tomdzk/rocksdb/db/fault_injection_test.cc:517 #2 ./fault_injection_test() [0x415bea] rocksdb::_Test_FaultTest::_RunIt() /data/users/tomdzk/rocksdb/db/fault_injection_test.cc:507 #3 ./fault_injection_test() [0x584367] rocksdb::test::RunAllTests() /data/users/tomdzk/rocksdb/util/testharness.cc:70 #4 /usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/libc.so.6(__libc_start_main+0x10e) [0x7f7a40857efe] ?? ??:0 #5 ./fault_injection_test() [0x408bb8] _start ??:0 ``` so I commented out the test invocation in the source code for now (lines 514-520) so it can be merged. Test Plan: This is a new test. Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31587
-
- 08 1月, 2015 2 次提交
-
-
由 Igor Canadi 提交于
Summary: I'm tired of double-tab when opening build_tools/<something>. This change will make bu<tab> fully complete my path :) Test Plan: `vi bu<tab>` gives me `vi build_tools/` yay! Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30639
-
由 Ameya Gupte 提交于
Summary: Create a benchmark for testing memtablereps. This diff is a bit rough, but it should do the trick until other bootcampers can clean it up. Addressing comments Removed the mutexes Changed ReadWriteBenchmark to fix number of reads and count the number of writes we can perform in that time. Test Plan: Run it. Below runs pass ./memtablerep_bench --benchmarks fillrandom,readrandom --memtablerep skiplist ./memtablerep_bench --benchmarks fillseq,readseq --memtablerep skiplist ./memtablerep_bench --benchmarks readwrite,seqreadwrite --memtablerep skiplist --num_operations 200 --num_threads 5 ./memtablerep_bench --benchmarks fillrandom,readrandom --memtablerep hashskiplist ./memtablerep_bench --benchmarks fillseq,readseq --memtablerep hashskiplist --num_scans 2 ./memtablerep_bench --benchmarks fillseq,readseq --memtablerep vector Reviewers: jpaton, ikabiljo, sdong Reviewed By: sdong Subscribers: dhruba, ameyag Differential Revision: https://reviews.facebook.net/D22683
-
- 24 12月, 2014 1 次提交
-
-
由 Manish Patil 提交于
Summary: Added necessary routines for dumping block based SST with block filter Test Plan: Added "raw" mode to utility sst_dump Reviewers: sdong, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D29679
-
- 23 12月, 2014 1 次提交
-
-
由 Lei Jin 提交于
Summary: Add support to allow nested config for block-based table factory. The format looks like this: "write_buffer_size=1024;block_based_table_factory={block_size=4k};max_write_buffer_num=2" Test Plan: unit test Reviewers: yhchiang, rven, igor, ljin, jonahcohen Reviewed By: jonahcohen Subscribers: jonahcohen, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29223
-
- 06 12月, 2014 2 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add -fno-exceptions flag to ROCKSDB_LITE. Test Plan: make OPT=-DROCKSDB_LITE shared_lib -j32 Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29901
-
由 Venkatesh Radhakrishnan 提交于
Summary: While running rocksdb tests, we sometimes encounter errors and the test run stops. We now provide a new make target call check_some which restarts the test run from a specific test and continues from there depending on the value of the environment variable ROCKSDBTESTS_START Test Plan: Run make check_some with different values of ROCKSDBTESTS_START. Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29913
-
- 02 12月, 2014 1 次提交
-
-
由 fyrz 提交于
-
- 01 12月, 2014 1 次提交
-
-
由 Stefan Eilemann 提交于
-
- 21 11月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add GetThreadList API, which allows developer to track the status of each process. Currently, calling GetThreadList will only get the list of background threads in RocksDB with their thread-id and thread-type (priority) set. Will add more support on this in the later diffs. ThreadStatus currently has the following properties: // An unique ID for the thread. const uint64_t thread_id; // The type of the thread, it could be ROCKSDB_HIGH_PRIORITY, // ROCKSDB_LOW_PRIORITY, and USER_THREAD const ThreadType thread_type; // The name of the DB instance where the thread is currently // involved with. It would be set to empty string if the thread // does not involve in any DB operation. const std::string db_name; // The name of the column family where the thread is currently // It would be set to empty string if the thread does not involve // in any column family. const std::string cf_name; // The event that the current thread is involved. // It would be set to empty string if the information about event // is not currently available. Test Plan: ./thread_list_test export ROCKSDB_TESTS=GetThreadList ./db_test Reviewers: rven, igor, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D25047
-
- 20 11月, 2014 2 次提交
-
-
由 fyrz 提交于
Currently maven publishing uses the library with debug symbols. What leads to unnecessary big library sizes. Included strip to remove unnecessary stuff. 40M -> 2.7M
-
由 fyrz 提交于
Previous to this commit too much targets got dependencies on javadocs target. Introduced one additional target "javalib" which resolves that situation. JavaDoc will now be generated once while executing a task with prefix "rocksdbjava".
-
- 15 11月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: This is just a simple test that passes two files though a compaction. It shows the framework so that people can continue building new compaction *unit* tests. In the future we might want to move some Compaction* tests from DBTest here. For example, CompactBetweenSnapshot seems a good candidate. Hopefully this test can be simpler when we mock out VersionSet. Test Plan: this is a test Reviewers: ljin, rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28449
-