- 31 12月, 2015 2 次提交
-
-
由 Islam AbdelRahman 提交于
fix vfunc inconsistency between env_hdfs and env
-
由 Shu Zhang 提交于
-
- 30 12月, 2015 5 次提交
-
-
由 Shu Zhang 提交于
-
由 Venkatesh Radhakrishnan 提交于
Summary: Fix CLANG build error caused by type mismatch. Changed type to size_t. Test Plan: Clang build and make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52413
-
由 sdong 提交于
Summary: TSAN fails on DynamicBloomTest.concurrent_with_perf. This change fixes it. Not sure why though. Test Plan: Run the test with TSAN and make sure no warning shown. Reviewers: yhchiang, IslamAbdelRahman, anthony, ngbronson, rven Reviewed By: rven Subscribers: rven, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52383
-
由 Venkatesh Radhakrishnan 提交于
Summary: This is an initial diff for providing the ability to delete files which are completely within a given range of keys. Test Plan: DBCompactionTest.DeleteRange Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52293
-
由 sdong 提交于
Summary: After removing two move operations, we can make CLANG 3.7 build pass under GCC 4.8.1. Test Plan: USE_CLANG=1 ROCKSDB_FBCODE_BUILD_WITH_481=1 make all -j32 Reviewers: yhchiang, IslamAbdelRahman, rven, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52365
-
- 29 12月, 2015 7 次提交
-
-
由 sdong 提交于
Summary: Adding "__attribute__((__unused__))" after padding fields will pass CLANG build but will fail gcc 4.8.1. Fix it by not generating it under GCC 4.8.1. Test Plan: Build under four combinations of USE_CLANG=0,1 and ROCKSDB_FBCODE_BUILD_WITH_481=0.1. Reviewers: yhchiang, rven, ngbronson, anthony, IslamAbdelRahman Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52371
-
由 Islam AbdelRahman 提交于
Disable Visual Studio Warning C4351 and Add some type casts to suppress warnings
-
由 Siying Dong 提交于
Currently Windows build is broken because of Warning C4351. Disable the warning before figuring out the right way to fix it.
-
由 sdong 提交于
Summary: ColumnFamilyTest.WriteStallSingleColumnFamily and ColumnFamilyTest.WriteStallTwoColumnFamilies didn't clean up test state cleanly, causing memory leak. Fix it. Test Plan: Run the two tests in valgrind and make sure they now pass. Reviewers: yhchiang, anthony, rven, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52347
-
由 sdong 提交于
Summary: 7d87f027 changed WriteBatch::InsertInto(). Need to change it. Test Plan: Run it. Reviewers: yhchiang, ngbronson Reviewed By: ngbronson Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52353
-
由 sdong 提交于
Summary: Fix some CLANG errors introduced in 7d87f027 Test Plan: Build with both of CLANG and gcc Reviewers: rven, yhchiang, kradhakrishnan, anthony, IslamAbdelRahman, ngbronson Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52329
-
由 Siying Dong 提交于
add a factory method for creating hdfs env
-
- 27 12月, 2015 2 次提交
- 26 12月, 2015 2 次提交
-
-
由 Nathan Bronson 提交于
Summary: This diff adds support for concurrent adds to the skiplist memtable implementations. Memory allocation is made thread-safe by the addition of a spinlock, with small per-core buffers to avoid contention. Concurrent memtable writes are made via an additional method and don't impose a performance overhead on the non-concurrent case, so parallelism can be selected on a per-batch basis. Write thread synchronization is an increasing bottleneck for higher levels of concurrency, so this diff adds --enable_write_thread_adaptive_yield (default off). This feature causes threads joining a write batch group to spin for a short time (default 100 usec) using sched_yield, rather than going to sleep on a mutex. If the timing of the yield calls indicates that another thread has actually run during the yield then spinning is avoided. This option improves performance for concurrent situations even without parallel adds, although it has the potential to increase CPU usage (and the heuristic adaptation is not yet mature). Parallel writes are not currently compatible with inplace updates, update callbacks, or delete filtering. Enable it with --allow_concurrent_memtable_write (and --enable_write_thread_adaptive_yield). Parallel memtable writes are performance neutral when there is no actual parallelism, and in my experiments (SSD server-class Linux and varying contention and key sizes for fillrandom) they are always a performance win when there is more than one thread. Statistics are updated earlier in the write path, dropping the number of DB mutex acquisitions from 2 to 1 for almost all cases. This diff was motivated and inspired by Yahoo's cLSM work. It is more conservative than cLSM: RocksDB's write batch group leader role is preserved (along with all of the existing flush and write throttling logic) and concurrent writers are blocked until all memtable insertions have completed and the sequence number has been advanced, to preserve linearizability. My test config is "db_bench -benchmarks=fillrandom -threads=$T -batch_size=1 -memtablerep=skip_list -value_size=100 --num=1000000/$T -level0_slowdown_writes_trigger=9999 -level0_stop_writes_trigger=9999 -disable_auto_compactions --max_write_buffer_number=8 -max_background_flushes=8 --disable_wal --write_buffer_size=160000000 --block_size=16384 --allow_concurrent_memtable_write" on a two-socket Xeon E5-2660 @ 2.2Ghz with lots of memory and an SSD hard drive. With 1 thread I get ~440Kops/sec. Peak performance for 1 socket (numactl -N1) is slightly more than 1Mops/sec, at 16 threads. Peak performance across both sockets happens at 30 threads, and is ~900Kops/sec, although with fewer threads there is less performance loss when the system has background work. Test Plan: 1. concurrent stress tests for InlineSkipList and DynamicBloom 2. make clean; make check 3. make clean; DISABLE_JEMALLOC=1 make valgrind_check; valgrind db_bench 4. make clean; COMPILE_WITH_TSAN=1 make all check; db_bench 5. make clean; COMPILE_WITH_ASAN=1 make all check; db_bench 6. make clean; OPT=-DROCKSDB_LITE make check 7. verify no perf regressions when disabled Reviewers: igor, sdong Reviewed By: sdong Subscribers: MarkCallaghan, IslamAbdelRahman, anthony, yhchiang, rven, sdong, guyg8, kradhakrishnan, dhruba Differential Revision: https://reviews.facebook.net/D50589
-
由 sdong 提交于
Summary: DBTest.HardLimit fails in appveyor build. Use special mem table to make the test behavior depends less on platform Test Plan: Run the test with JEMALLOC both on and off. Reviewers: yhchiang, kradhakrishnan, rven, anthony, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52317
-
- 25 12月, 2015 2 次提交
- 24 12月, 2015 14 次提交
-
-
由 Siying Dong 提交于
Enable MS compiler warning c4244.
-
由 Siying Dong 提交于
Fix clang warnings
-
由 Siying Dong 提交于
Fix computation of size of last sub-compaction
-
由 sdong 提交于
Summary: DBTest.DelayedWriteRate has sign and unsign comparisons that break Windows build. Fix it. Test Plan: Build and run the test modified. Reviewers: IslamAbdelRahman, rven, anthony, yhchiang, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52311
-
由 sdong 提交于
Summary: Warning in release build. Test Plan: Make release and make all Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52305
-
由 Zhipeng Jia 提交于
-
由 Shu Zhang 提交于
-
由 Andrew Kryczka 提交于
Summary: It was already built in third-party2 but the include/library paths in rocksdb hadn't been updated accordingly. Test Plan: verified build works $ make clean && make -j32 all $ make clean && USE_CLANG=1 make -j32 all Reviewers: cyan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52299
-
由 sdong 提交于
Summary: We now have a mechanism to further slowdown writes. Double default options.delayed_write_rate to try to keep the default behavior closer to it used to be. Test Plan: Run all tests. Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: yhchiang, kradhakrishnan, rven, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52281
-
由 Zhipeng Jia 提交于
-
由 sdong 提交于
Summary: It's usually hard for users to set a value of options.delayed_write_rate. With this diff, after slowdown condition triggers, we greedily reduce write rate if estimated pending compaction bytes increase. If estimated compaction pending bytes drop, we increase the write rate. Test Plan: Add a unit test Test with db_bench setting: TEST_TMPDIR=/dev/shm/ ./db_bench --benchmarks=fillrandom -num=10000000 --soft_pending_compaction_bytes_limit=1000000000 --hard_pending_compaction_bytes_limit=3000000000 --delayed_write_rate=100000000 and make sure without the commit, write stop will happen, but with the commit, it will not happen. Reviewers: igor, anthony, rven, yhchiang, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52131
-
由 Andrew Kryczka 提交于
Summary: Missed this in https://reviews.facebook.net/D51633 because I didn't wait for 'make commit-prereq' to finish Test Plan: make clean && USE_CLANG=1 make -j32 all Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52275
-
由 Andrew Kryczka 提交于
Summary: When Get() or NewIterator() trigger file loads, skip caching the filter block if (1) optimize_filters_for_hits is set and (2) the file is on the bottommost level. Also skip checking filters under the same conditions, which means that for a preloaded file or a file that was trivially-moved to the bottom level, its filter block will eventually expire from the cache. - added parameters/instance variables in various places in order to propagate the config ("skip_filters") from version_set to block_based_table_reader - in BlockBasedTable::Rep, this optimization prevents filter from being loaded when the file is opened simply by setting filter_policy = nullptr - in BlockBasedTable::Get/BlockBasedTable::NewIterator, this optimization prevents filter from being used (even if it was loaded already) by setting filter = nullptr Test Plan: updated unit test: $ ./db_test --gtest_filter=DBTest.OptimizeFiltersForHits will also run 'make check' Reviewers: sdong, igor, paultuckfield, anthony, rven, kradhakrishnan, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D51633
-
由 Siying Dong 提交于
Fix clang warning
-
- 23 12月, 2015 5 次提交
-
-
由 Zhipeng Jia 提交于
-
由 Siying Dong 提交于
Make Status moveable
-
由 Dmitri Smirnov 提交于
Status is a class which is frequently returned by value from functions. Making it movable avoids 99% of the copies automatically on return by value.
-
由 Islam AbdelRahman 提交于
Summary: Fix compiling under ROCKSDB_LITE Test Plan: OPT="-DROCKSDB_LITE" make -j64 check make check -j64 Reviewers: rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52239
-
由 Islam AbdelRahman 提交于
Summary: Add CompactionReason to CompactionJobInfo This will allow users to understand why compaction started which will help options tuning Test Plan: added new tests make check -j64 Reviewers: yhchiang, anthony, kradhakrishnan, sdong, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51975
-
- 22 12月, 2015 1 次提交
-
-
由 Zhipeng Jia 提交于
-