- 07 1月, 2016 9 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: This patch addes ColumnFamilyHandle::GetDescriptor(), which allows developers to obtain the CF options and names of the associated column family given its handle. // Returns the up-to-date descriptor used by the current handle. Since it // returns the up-to-date information, this call might internally locks // and releases DB mutex to access the up-to-date CF options. virtual ColumnFamilyDescriptor GetDescriptor() = 0; Test Plan: augment column_family_test Reviewers: sdong, yoshinorim, IslamAbdelRahman, rven, kradhakrishnan, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51543
-
由 agiardullo 提交于
Summary: fix memory leak in test code Test Plan: ran test Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52617
-
由 Gunnar Kudrjavets 提交于
Summary: This set of changes implements the following design: `ldb` will utilize `--path` parameter which can be used to specify a file name. Tool will then apply some heuristic to determine how to output the data properly. The design decision is not to probe the file content, but use file names to determine what dumping function to call. Usage examples: Understands that path points to a manifest file and dumps it. `./ldb --path=/tmp/test_db/MANIFEST-000023 dump` Understands that path points to a WAL file and dumps it. `./ldb --path=/tmp/test_db/000024.log dump --header` Understands that path points to a SST file and dumps it. `./ldb --path=/tmp/test_db/000007.sst dump` Figures out that none of the supported file types are applicable and outputs an appropriate error message. `./ldb --path=/tmp/cron.log dump` Test Plan: Basics: git diff make clean make -j 32 commit-prereq arc lint More specific testing (done as part of commit-prereq, but can be iterated separately when making isolated changes): make clean make ldb python tools/ldb_test.py make rocksdb_dump make rocksdb_undump sh tools/rocksdb_dump_test.sh Reviewers: rven, IslamAbdelRahman, yhchiang, kradhakrishnan, anthony, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52269
-
由 Igor Canadi 提交于
Fix index seeking in BlockTableReader::PrefixMayMatch.
-
由 Reid Horuff 提交于
Summary: DBImpl::GetLatestSequenceForKey() can do memcpy's to load a value that will never be used. This can be optimized by changing all the Get() functions called to optionally not fetch the value (and only fetch the sequencenumber). Test Plan: optimistic_transaction_test and transaction_test Reviewers: anthony Reviewed By: anthony Subscribers: leveldb, dhruba, hermanlee4 Differential Revision: https://reviews.facebook.net/D52227
-
由 Peter Mattis 提交于
PrefixMayMatch previously seeked in the prefix index using an internal key with a sequence number of 0. This would cause the prefix index seek to fall off the end if the last key in the index had a user-key greater than or equal to the key being looked for. Falling off the end of the index in turn results in PrefixMayMatch returning false if the index is in memory.
-
由 Igor Canadi 提交于
Summary: See a bug report here: https://github.com/facebook/rocksdb/issues/921 The fix is to not check the shared/ directory if share_table_files is false. We could also check FileExists() before GetChildren(), but that will add extra latency when Env is Hdfs :( Test Plan: added a unit test Reviewers: rven, sdong, IslamAbdelRahman, yhchiang, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52593
-
由 agiardullo 提交于
Summary: Need to make sure the background task gets scheduled before it goes out of scope. Test Plan: ran test. Will see if sporadic valgrind failures go away. Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52575
-
由 agiardullo 提交于
Summary: Make sure SleepingTask has bene run before it goes out of scope. Test Plan: run test Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52581
-
- 06 1月, 2016 1 次提交
-
-
由 Igor Canadi 提交于
Fix failing assertion in logger on Windows when the disk is full.
-
- 05 1月, 2016 7 次提交
-
-
由 Marek Kurdej 提交于
-
由 Jay Edgar 提交于
Summary: If block_restart_interval gets set to less than 1 an assert will be triggered in BlockBuilder::BlockBuilder(). This prevents the user from doing this by silently setting any value less than 1 to 1. Test Plan: Added a test (in BlockBasedTableTest in table_test) that checks invalid values to make sure that they are reset to the expected values. The block_restart_interval value is checked along with block_size_deviation which also silently sets the value if it is outside a specific range. Reviewers: yoshinorim, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52509
-
由 Mark Callaghan 提交于
Summary: 1) changes tools/{benchmark,run_flash_bench}.sh to optionally use the write rate limit 2) removes code for --writes_per_second and switches the 'background' write rate limit to use --benchmark_write_rate_limit Replaces https://reviews.facebook.net/D49113 Task ID: #9555881 Blame Rev: Test Plan: tools/run_flash_bench.sh Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52485
-
由 Igor Canadi 提交于
Summary: as title Test Plan: none Reviewers: sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52503
-
由 Venkatesh Radhakrishnan 提交于
Summary: We need to clean the job context if we end up not deleting any files because no files are in the range specified. Test Plan: DBCompactionTest.DeleteFileRange Reviewers: sdong, anthony, yhchiang, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52467
-
由 Igor Canadi 提交于
Expose the option, memtable_prefix_bloom_huge_page_tlb_size, via C API
-
由 Igor Canadi 提交于
Fix for #909: Support creation of "full" format bloom filter from C API
-
- 31 12月, 2015 6 次提交
-
-
由 Warren Falk 提交于
-
由 Warren Falk 提交于
-
由 Islam AbdelRahman 提交于
Summary: GTEST dont compile under clang when -Werror=missing-field-initializers is set revert this change Test Plan: USE_CLANG=1 make check make check Reviewers: rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52455
-
由 Nathan Bronson 提交于
Summary: myrocks seems to build rocksdb using -Wmissing-field-initializers (and treats warnings as errors). This diff adds that flag to the rocksdb build, and fixes the compilation failures that result. I have not checked for any other differences in the build flags for rocksdb build as part of myrocks. Test Plan: make check Reviewers: sdong, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52443
-
由 Islam AbdelRahman 提交于
fix vfunc inconsistency between env_hdfs and env
-
由 Shu Zhang 提交于
-
- 30 12月, 2015 5 次提交
-
-
由 Shu Zhang 提交于
-
由 Venkatesh Radhakrishnan 提交于
Summary: Fix CLANG build error caused by type mismatch. Changed type to size_t. Test Plan: Clang build and make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52413
-
由 sdong 提交于
Summary: TSAN fails on DynamicBloomTest.concurrent_with_perf. This change fixes it. Not sure why though. Test Plan: Run the test with TSAN and make sure no warning shown. Reviewers: yhchiang, IslamAbdelRahman, anthony, ngbronson, rven Reviewed By: rven Subscribers: rven, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52383
-
由 Venkatesh Radhakrishnan 提交于
Summary: This is an initial diff for providing the ability to delete files which are completely within a given range of keys. Test Plan: DBCompactionTest.DeleteRange Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52293
-
由 sdong 提交于
Summary: After removing two move operations, we can make CLANG 3.7 build pass under GCC 4.8.1. Test Plan: USE_CLANG=1 ROCKSDB_FBCODE_BUILD_WITH_481=1 make all -j32 Reviewers: yhchiang, IslamAbdelRahman, rven, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52365
-
- 29 12月, 2015 7 次提交
-
-
由 sdong 提交于
Summary: Adding "__attribute__((__unused__))" after padding fields will pass CLANG build but will fail gcc 4.8.1. Fix it by not generating it under GCC 4.8.1. Test Plan: Build under four combinations of USE_CLANG=0,1 and ROCKSDB_FBCODE_BUILD_WITH_481=0.1. Reviewers: yhchiang, rven, ngbronson, anthony, IslamAbdelRahman Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52371
-
由 Islam AbdelRahman 提交于
Disable Visual Studio Warning C4351 and Add some type casts to suppress warnings
-
由 Siying Dong 提交于
Currently Windows build is broken because of Warning C4351. Disable the warning before figuring out the right way to fix it.
-
由 sdong 提交于
Summary: ColumnFamilyTest.WriteStallSingleColumnFamily and ColumnFamilyTest.WriteStallTwoColumnFamilies didn't clean up test state cleanly, causing memory leak. Fix it. Test Plan: Run the two tests in valgrind and make sure they now pass. Reviewers: yhchiang, anthony, rven, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52347
-
由 sdong 提交于
Summary: 7d87f027 changed WriteBatch::InsertInto(). Need to change it. Test Plan: Run it. Reviewers: yhchiang, ngbronson Reviewed By: ngbronson Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52353
-
由 sdong 提交于
Summary: Fix some CLANG errors introduced in 7d87f027 Test Plan: Build with both of CLANG and gcc Reviewers: rven, yhchiang, kradhakrishnan, anthony, IslamAbdelRahman, ngbronson Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52329
-
由 Siying Dong 提交于
add a factory method for creating hdfs env
-
- 27 12月, 2015 2 次提交
- 26 12月, 2015 2 次提交
-
-
由 Nathan Bronson 提交于
Summary: This diff adds support for concurrent adds to the skiplist memtable implementations. Memory allocation is made thread-safe by the addition of a spinlock, with small per-core buffers to avoid contention. Concurrent memtable writes are made via an additional method and don't impose a performance overhead on the non-concurrent case, so parallelism can be selected on a per-batch basis. Write thread synchronization is an increasing bottleneck for higher levels of concurrency, so this diff adds --enable_write_thread_adaptive_yield (default off). This feature causes threads joining a write batch group to spin for a short time (default 100 usec) using sched_yield, rather than going to sleep on a mutex. If the timing of the yield calls indicates that another thread has actually run during the yield then spinning is avoided. This option improves performance for concurrent situations even without parallel adds, although it has the potential to increase CPU usage (and the heuristic adaptation is not yet mature). Parallel writes are not currently compatible with inplace updates, update callbacks, or delete filtering. Enable it with --allow_concurrent_memtable_write (and --enable_write_thread_adaptive_yield). Parallel memtable writes are performance neutral when there is no actual parallelism, and in my experiments (SSD server-class Linux and varying contention and key sizes for fillrandom) they are always a performance win when there is more than one thread. Statistics are updated earlier in the write path, dropping the number of DB mutex acquisitions from 2 to 1 for almost all cases. This diff was motivated and inspired by Yahoo's cLSM work. It is more conservative than cLSM: RocksDB's write batch group leader role is preserved (along with all of the existing flush and write throttling logic) and concurrent writers are blocked until all memtable insertions have completed and the sequence number has been advanced, to preserve linearizability. My test config is "db_bench -benchmarks=fillrandom -threads=$T -batch_size=1 -memtablerep=skip_list -value_size=100 --num=1000000/$T -level0_slowdown_writes_trigger=9999 -level0_stop_writes_trigger=9999 -disable_auto_compactions --max_write_buffer_number=8 -max_background_flushes=8 --disable_wal --write_buffer_size=160000000 --block_size=16384 --allow_concurrent_memtable_write" on a two-socket Xeon E5-2660 @ 2.2Ghz with lots of memory and an SSD hard drive. With 1 thread I get ~440Kops/sec. Peak performance for 1 socket (numactl -N1) is slightly more than 1Mops/sec, at 16 threads. Peak performance across both sockets happens at 30 threads, and is ~900Kops/sec, although with fewer threads there is less performance loss when the system has background work. Test Plan: 1. concurrent stress tests for InlineSkipList and DynamicBloom 2. make clean; make check 3. make clean; DISABLE_JEMALLOC=1 make valgrind_check; valgrind db_bench 4. make clean; COMPILE_WITH_TSAN=1 make all check; db_bench 5. make clean; COMPILE_WITH_ASAN=1 make all check; db_bench 6. make clean; OPT=-DROCKSDB_LITE make check 7. verify no perf regressions when disabled Reviewers: igor, sdong Reviewed By: sdong Subscribers: MarkCallaghan, IslamAbdelRahman, anthony, yhchiang, rven, sdong, guyg8, kradhakrishnan, dhruba Differential Revision: https://reviews.facebook.net/D50589
-
由 sdong 提交于
Summary: DBTest.HardLimit fails in appveyor build. Use special mem table to make the test behavior depends less on platform Test Plan: Run the test with JEMALLOC both on and off. Reviewers: yhchiang, kradhakrishnan, rven, anthony, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52317
-
- 25 12月, 2015 1 次提交
-
-
由 Shu Zhang 提交于
-