- 18 7月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: CYGWIN avoided fread_unlocked in a wrong way. Fix it to the standard way. Test Plan: Run tests Reviewers: anthony, kradhakrishnan, IslamAbdelRahman, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42549
-
- 14 7月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: This helps Windows port to format their changes, as discussed. Might have formatted some other codes too becasue last 10 commits include more. Test Plan: Build it. Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D41961
-
- 03 7月, 2015 1 次提交
-
-
由 Dmitri Smirnov 提交于
-
- 02 7月, 2015 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Make RocksDb build and run on Windows to be functionally complete and performant. All existing test cases run with no regressions. Performance numbers are in the pull-request. Test plan: make all of the existing unit tests pass, obtain perf numbers. Co-authored-by: Praveen Rao praveensinghrao@outlook.com Co-authored-by: Sherlock Huang baihan.huang@gmail.com Co-authored-by: Alex Zinoviev alexander.zinoviev@me.com Co-authored-by: Dmitri Smirnov dmitrism@microsoft.com
-
- 23 6月, 2015 1 次提交
-
-
由 krad 提交于
Summary: MyRocks need a mechanism to track read outliers. We need to expose this stat. Test Plan: None Reviewers: sdong CC: leveldb Task ID: #7152512 Blame Rev:
-
- 18 6月, 2015 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Before this patch, any function call to ThreadStatusUtil might automatically initialize and register the thread status data. However, if it is the user-thread making this call, the allocated thread-status-data will never be released as such threads are not managed by rocksdb. In this patch, I remove the automatic-initialization part. Thread-status data is only initialized and uninitialized in Env during the thread creation and destruction. Test Plan: db_test thread_list_test listener_test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40017
-
- 12 6月, 2015 2 次提交
-
-
由 Islam AbdelRahman 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add Env::GetThreadID(), which returns the ID of the current thread. In addition, make GetThreadList() and InfoLog use same unique ID for the same thread. Test Plan: db_test listener_test Reviewers: igor, rven, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39735
-
- 02 6月, 2015 1 次提交
-
-
由 Mike Kolupaev 提交于
Summary: We occasionally get write stalls (>1s Write() calls) on HDD under read load. The following timers explain almost all of the stalls: - perf_context.db_mutex_lock_nanos - perf_context.db_condition_wait_nanos - iostats_context.open_time - iostats_context.allocate_time - iostats_context.write_time - iostats_context.range_sync_time - iostats_context.logger_time In my experiments each of these occasionally takes >1s on write path under some workload. There are rare cases when Write() takes long but none of these takes long. Test Plan: Added code to our application to write the listed timings to log for slow writes. They usually add up to almost exactly the time Write() call took. Reviewers: rven, yhchiang, sdong Reviewed By: sdong Subscribers: march, dhruba, tnovak Differential Revision: https://reviews.facebook.net/D39177
-
- 19 5月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: sync_file_range is not always asyncronous and thus can block writes if we do this for WAL in the foreground thread. See more here: http://yoshinorimatsunobu.blogspot.com/2014/03/how-syncfilerange-really-works.html Some users don't want us to call sync_file_range on WALs. Some other do. Thus, I'm adding a separate option wal_bytes_per_sync to control calling sync_file_range on WAL files. bytes_per_sync will apply only to table files now. Test Plan: no more sync_file_range for WAL as evidenced by strace Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D38253
-
- 24 4月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: Make it build for CYGWIN. Need to define "-std=gnu++11" instead of "-std=c++11" and use some replacement functions. Test Plan: Build it and run some unit tests in CYGWIN Reviewers: yhchiang, rven, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37605
-
- 19 3月, 2015 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Right now if they system we are compiling on is not Linux and not Mac we will get a compilation error this diff use chrono as a fallback when we are compiling on something other than Linux/FreeBSD/Mac Test Plan: compile on CentOS/FreeBSD ./db_test (still running) Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35277
-
- 17 3月, 2015 2 次提交
-
-
由 Danny Al-Gaaf 提交于
Fix '#elif with no expression' add defined() to check. Signed-off-by: NDanny Al-Gaaf <danny.al-gaaf@bisect.de>
-
由 Venkatesh Radhakrishnan 提交于
Summary: On RocksDB, when there are multiple instances doing flushes/compactions in the background, the close call takes a long time because the flushes/compactions need to complete before the database can shut down. If another instance is using the background threads and the compaction for this instance is in the queue since it has been scheduled, we still cannot shutdown. We now remove the scheduled background tasks which have not yet started running, so that shutdown is speeded up. Test Plan: DB Test added. Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33741
-
- 04 3月, 2015 1 次提交
-
-
由 Igor Canadi 提交于
Summary: For some reason, libstdc++ implements steady_clock::now() using syscall instead of VDSO optimized clock_gettime() when using glibc 2.16 and earlier. This leads to significant performance degradation for users with older glibcs. See bug reported here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59177 We observed this behavior when testing mongo on AWS hosts. Facebook hosts are unaffected since we use glibc2.17 and 2.20. Revert "Fix timing" This reverts commit 965d9d50. Revert "Use chrono for timing" This reverts commit 001ce64d. Test Plan: make check Reviewers: MarkCallaghan, yhchiang, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34371
-
- 27 2月, 2015 1 次提交
-
-
由 Igor Sugak 提交于
Summary: When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Almost all of them are missing override errors. This diff adds missing override keyword. No manual changes. Prerequisites: bear and clang 3.5 build with extra tools ```lang=bash % USE_CLANG=1 bear make all # generate a compilation database http://clang.llvm.org/docs/JSONCompilationDatabase.html % clang-modernize -p . -include . -add-override % make format ``` Test Plan: Make sure all tests are passing. ```lang=bash % #Use default fb code clang. % make check ``` Verify less error and no missing override errors. ```lang=bash % # Have trunk clang present in path. % ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make ``` Reviewers: igor, kradhakrishnan, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34077
-
- 18 2月, 2015 1 次提交
-
-
由 Igor Sugak 提交于
Summary: scan-build complains with division by zero warning in a test. Added an assertion to prevent this. scan-build report: http://home.fburl.com/~sugak/latest6/report-c61be9.html#EndPath Test Plan: Make sure scan-build does not report 'Division by zero' and all tests are passing. ```lang=bash % make analyze % make check ``` Reviewers: igor, meyering Reviewed By: meyering Subscribers: sdong, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33495
-
- 02 2月, 2015 1 次提交
-
-
由 Erik Garrison 提交于
It doesn't seem this is needed.
-
- 31 12月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add structures for exposing events and operations. Event describes high-level action about a thread such as doing compaciton or doing flush, while an operation describes lower-level action of a thread such as reading / writing a SST table, waiting for mutex. Events and operations are designed to be independent. One thread would typically involve in one event and one operation. Code instrument will be in a separate diff. Test Plan: Add unit-tests in thread_list_test make dbg -j32 ./thread_list_test export ROCKSDB_TESTS=ThreadList ./db_test Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: rven, jonahcohen, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D29781
-
- 23 12月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: GetThreadList() feature depends on the thread creation and destruction, which is currently handled under Env. This patch moves GetThreadList() feature under Env to better manage the dependency of GetThreadList() feature on thread creation and destruction. Renamed ThreadStatusImpl to ThreadStatusUpdater. Add ThreadStatusUtil, which is a static class contains utility functions for ThreadStatusUpdater. Test Plan: run db_test, thread_list_test and db_bench and verify the life cycle of Env and ThreadStatusUpdater is properly managed. Reviewers: igor, sdong Reviewed By: sdong Subscribers: ljin, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30057
-
- 18 12月, 2014 1 次提交
-
-
由 Venkatesh Radhakrishnan 提交于
Summary: Release locks before calling exit. Test Plan: Force errors in debugger and verify correctness Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30423
-
- 10 12月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Why do we assert here? This doesn't seem like user friendly thing to do :) Test Plan: none Reviewers: sdong, yhchiang, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D30027
-
- 26 11月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
-
- 21 11月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Add GetThreadList API, which allows developer to track the status of each process. Currently, calling GetThreadList will only get the list of background threads in RocksDB with their thread-id and thread-type (priority) set. Will add more support on this in the later diffs. ThreadStatus currently has the following properties: // An unique ID for the thread. const uint64_t thread_id; // The type of the thread, it could be ROCKSDB_HIGH_PRIORITY, // ROCKSDB_LOW_PRIORITY, and USER_THREAD const ThreadType thread_type; // The name of the DB instance where the thread is currently // involved with. It would be set to empty string if the thread // does not involve in any DB operation. const std::string db_name; // The name of the column family where the thread is currently // It would be set to empty string if the thread does not involve // in any column family. const std::string cf_name; // The event that the current thread is involved. // It would be set to empty string if the information about event // is not currently available. Test Plan: ./thread_list_test export ROCKSDB_TESTS=GetThreadList ./db_test Reviewers: rven, igor, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D25047
-
- 15 11月, 2014 1 次提交
-
-
由 Venkatesh Radhakrishnan 提交于
Summary: Store links to live files in directory on same disk Test Plan: Take snapshot and open it. Added a test GetSnapshotLink in db_test. Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28713
-
- 14 11月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: So iOS size_t is 32-bit, so we need to static_cast<size_t> any uint64_t :( Test Plan: TARGET_OS=IOS make static_lib Reviewers: dhruba, ljin, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28743
-
- 12 11月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: We need to turn on -Wshorten-64-to-32 for mobile. See D1671432 (internal phabricator) for details. This diff turns on the warning flag and fixes all the errors. There were also some interesting errors that I might call bugs, especially in plain table. Going forward, I think it makes sense to have this flag turned on and be very very careful when converting 64-bit to 32-bit variables. Test Plan: compiles Reviewers: ljin, rven, yhchiang, sdong Reviewed By: yhchiang Subscribers: bobbaldwin, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D28689
-
- 04 11月, 2014 1 次提交
-
-
由 sdong 提交于
DB::Open() to automatically increase thread pool size if it is smaller than max number of parallel compactions or flushes Summary: With the patch, thread pool size will be automatically increased if DB's options ask for more parallelism of compactions or flushes. Too many users have been confused by the API. Change it to make it harder for users to make mistakes Test Plan: Add two unit tests to cover the function. Reviewers: yhchiang, rven, igor, MarkCallaghan, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27555
-
- 30 10月, 2014 2 次提交
-
-
由 Lei Jin 提交于
Summary: Revmoed this in D25641, causing compiler complain. put it back Test Plan: make release Reviewers: igor, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D27891
-
由 Lei Jin 提交于
Summary: ftruncate does not always free preallocated unused space at the end of file. In some cases, we pin too much disk space than it should Test Plan: env_test Reviewers: sdong, rven, yhchiang, igor Reviewed By: igor Subscribers: nkg-, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D25641
-
- 25 10月, 2014 2 次提交
-
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Since we depend on C++11, we might as well use it for timing, instead of this platform-depended code. Test Plan: Ran autovector_test, which reports time and confirmed that output is similar to master Reviewers: ljin, sdong, yhchiang, rven, dhruba Reviewed By: dhruba Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D25587
-
- 30 8月, 2014 1 次提交
-
-
由 Lei Jin 提交于
Summary: BlockBasedTable sst file size can grow to a large size when universal compaction is used. When index block exceeds 2G, pread seems to fail and return truncated data and causes "trucated block" error. I tried to use ``` #define _FILE_OFFSET_BITS 64 ``` But the problem still persists. Splitting a big write/read into smaller batches seems to solve the problem. Test Plan: successfully compacted a case with resulting sst file at ~90G (2.1G index block size) Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D22569
-
- 27 8月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: See https://github.com/facebook/rocksdb/issues/244#issuecomment-53372297 Also see this: https://github.com/facebook/rocksdb/blob/master/util/env_posix.cc#L1075 Test Plan: compiles Reviewers: yhchiang, ljin, sdong Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D22419
-
- 14 8月, 2014 2 次提交
-
-
由 ZHANG Biao 提交于
-
由 Lei Jin 提交于
Summary: This is a linux-specific system call. Test Plan: ran db_bench Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: haobo, leveldb Differential Revision: https://reviews.facebook.net/D21183
-
- 09 7月, 2014 1 次提交
-
-
由 Lei Jin 提交于
Summary: Add option and plugin rate limiter for PosixWritableFile. The rate limiter only applies to flush and compaction. WAL and MANIFEST are excluded from this enforcement. Test Plan: db_test Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19425
-
- 04 7月, 2014 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This diff allows the I/O stats about Flush and Compaction to be reported in a more accurate way. Instead of measuring the size of a file, it measure I/O cost in per read / write basis. Test Plan: make all check Reviewers: sdong, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19383
-
- 02 7月, 2014 1 次提交
-
-
由 Igor Canadi 提交于
Summary: https://www.facebook.com/groups/rocksdb.dev/permalink/590438347721350/ Test Plan: compiles Reviewers: sdong, ljin, yhchiang Reviewed By: yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19197
-
- 30 5月, 2014 1 次提交
-
-
由 sdong 提交于
Summary: https://github.com/facebook/rocksdb/commit/220132b65ec17abb037d3e79d5abf6ca8d797b96 correctly fixed the issue of thread ID printing when terminating a thread. Nothing wrong with it. This diff prints the ID in the same way as in PosixLogger::logv() so that users can be more easily to correlates them. Test Plan: run env_test and make sure it prints correctly. Reviewers: igor, haobo, ljin, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18819
-