- 30 10月, 2015 4 次提交
-
-
由 Siying Dong 提交于
Implement smart buffer management in Windows Env.
-
由 Islam AbdelRahman 提交于
Fix compilation problem on Windows.
-
由 Dmitri Smirnov 提交于
-
由 Dmitri Smirnov 提交于
char is not a valid template parameter for std::uniform_int_distribution according to the standard. Replacing with int should be just fine.
-
- 29 10月, 2015 7 次提交
-
-
由 Igor Canadi 提交于
Summary: We don't have access to GetLiveFilesMetadata() in RocksDB lite. If compiling write_stress for lite, I skip the check for leaked files, which depends on this function. Test Plan: OPT=-DROCKSDB_LITE m write_stress Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49647
-
由 Herman Lee 提交于
Summary: An uninitialized parameter was being passed into the call to fetch the table properties during the compaction notification callbacks. Test Plan: Build it with myrocks and verify unit test passed. Run unit tests. Reviewers: rven, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49635
-
由 Igor Canadi 提交于
Summary: The goal of this diff is to create a simple stress test with focus on catching: * bugs in compaction/flush processes, especially the ones that cause assertion errors * bugs in the code that deletes obsolete files There are two parts of the test: * write_stress, a binary that writes to the database * write_stress_runner.py, a script that invokes and kills write_stress Here are some interesting parts of write_stress: * Runs with very high concurrency of compactions and flushes (32 threads total) and tries to create a huge amount of small files * The keys written to the database are not uniformly distributed -- there is a 3-character prefix that mutates occasionally (in prefix mutator thread), in such a way that the first character mutates slower than second, which mutates slower than third character. That way, the compaction stress tests some interesting compaction features like trivial moves and bottommost level calculation * There is a thread that creates an iterator, holds it for couple of seconds and then iterates over all keys. This is supposed to test RocksDB's abilities to keep the files alive when there are references to them. * Some writes trigger WAL sync. This is stress testing our WAL sync code. * At the end of the run, we make sure that we didn't leak any of the sst files write_stress_runner.py changes the mode in which we run write_stress and also kills and restarts it. There are some interesting characteristics: * At the beginning we divide the full test runtime into smaller parts -- shorter runtimes (couple of seconds) and longer runtimes (100, 1000) seconds * The first time we run write_stress, we destroy the old DB. Every next time during the test, we use the same DB. * We can run in kill mode or clean-restart mode. Kill mode kills the write_stress violently. * We can run in mode where delete_obsolete_files_with_fullscan is true or false * We can run with low_open_files mode turned on or off. When it's turned on, we configure table cache to only hold a couple of files -- that way we need to reopen files every time we access them. Another goal was to create a stress test without a lot of parameters. So tools/write_stress_runner.py should only take one parameter -- runtime_sec and it should figure out everything else on its own. In a separate diff, I'll add this new test to our nightly legocastle runs. Test Plan: The goal of this test was to retroactively catch the following bugs: D33045, D48201, D46899, D42399. I failed to reproduce D48201, but all others have been caught! When i reverted https://reviews.facebook.net/D33045: ./write_stress --runtime_sec=200 --low_open_files_mode=true Iterator statuts not OK: IO error: /fast-rocksdb-tmp/rocksdb_test/write_stress/089166.sst: No such file or directory When i reverted https://reviews.facebook.net/D42399: python tools/write_stress_runner.py --runtime_sec=5000 Running write_stress, will kill after 5 seconds: ./write_stress --runtime_sec=-1 Running write_stress, will kill after 2 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --delete_obsolete_files_with_fullscan=true Running write_stress, will kill after 7 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false Running write_stress, will kill after 5 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false Running write_stress, will kill after 8 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --low_open_files_mode=true Write to DB failed: IO error: /fast-rocksdb-tmp/rocksdb_test/write_stress/019250.sst: No such file or directory ERROR: write_stress died with exitcode=-6 When i reverted https://reviews.facebook.net/D46899: python tools/write_stress_runner.py --runtime_sec=1000 runtime: 1000 Going to execute write stress for [3, 3, 100, 3, 2, 100, 1, 788] Running write_stress for 3 seconds: ./write_stress --runtime_sec=3 --low_open_files_mode=true Running write_stress for 3 seconds: ./write_stress --runtime_sec=3 --destroy_db=false --delete_obsolete_files_with_fullscan=true Running write_stress, will kill after 100 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --delete_obsolete_files_with_fullscan=true write_stress: db/db_impl.cc:2070: void rocksdb::DBImpl::MarkLogsSynced(uint64_t, bool, const rocksdb::Status&): Assertion `log.getting_synced' failed. ERROR: write_stress died with exitcode=-6 Reviewers: IslamAbdelRahman, yhchiang, rven, kradhakrishnan, sdong, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49533
-
由 sdong 提交于
Summary: include/posix/io_posix.h is not a public API. Although include/posix/ is not a public header directory, it is confusing to put non-public headers to under include/. Move it to util/ to be clearer. Test Plan: Run all tests Reviewers: rven, IslamAbdelRahman, anthony, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49611
-
由 Siying Dong 提交于
Fix MockTable ID storage
-
由 Dmitri Smirnov 提交于
On Windows two tests fail that use MockTable: flush_job_test and compaction_job_test with the following message: compaction_job_test_je.exe : Assertion failed: result.size() == 4, file c:\dev\rocksdb\rocksdb\table\mock_table.cc, line 110 Investigation reveals that this failure occurs when a 4 byte ID written to a beginning of the physically open file (main contents remains in a in-memory map) can not be read back. The reason for the failure is that the ID is written directly to a WritableFile bypassing WritableFileWriter. The side effect of that is that pending_sync_ never becomes true so the file is never flushed, however, the direct cause of the failure is that the filesize_ member of the WritableFileWriter remains zero. At Close() the file is truncated to that size and the file becomes empty so the ID can not be read back.
-
- 28 10月, 2015 8 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: It looks like WritableFileWriter::Append() was returning OK() even when there is an error Test Plan: make check Reviewers: sdong, yhchiang, anthony, rven, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49569
-
由 Siying Dong 提交于
Fix unused parameter warnings in db.h
-
由 sdong 提交于
Summary: include/posix/io_posix.h should not depend on ROCKSDB_FALLOCATE_PRESENT. Remove it. Test Plan: Build it with both of ROCKSDB_FALLOCATE_PRESENT defined and not defined. Reviewers: rven, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49563
-
由 Dmitri Smirnov 提交于
introduce a new DBOption random_access_max_buffer_size to limit the size of the random access buffer used for unbuffered access. Implement read ahead buffering when enabled. To that effect propagate compaction_readahead_size and the new option to the env options to make it available for the implementation. Add Hint() override so SetupForCompaction() call would call Hint() readahead can now be setup from both Hint() and EnableReadAhead() Add new option random_access_max_buffer_size support db_bench, options_helper to make it string parsable and the unit test.
-
由 sdong 提交于
Summary: Mac build breaks as include/posix/io_posix.h doesn't include errno. Move the exact function declaration to io_posix.cc Test Plan: Run all test. Will run on Mac Reviewers: rven, anthony, yhchiang, IslamAbdelRahman, igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49551
-
由 Siying Dong 提交于
Adding wal filter to inspect and filter wal records on recovery
-
由 sdong 提交于
Summary: crash_test still has a very low chance to hit some crash point. Have another mode for covering them more likely. Test Plan: Run crash_test and see db_stress is called with expected prameters. Reviewers: kradhakrishnan, igor, anthony, rven, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49473
-
由 Igor Canadi 提交于
Error while cmake by building from zip-archive
-
- 27 10月, 2015 8 次提交
-
-
由 Praveen Rao 提交于
-
由 Praveen Rao 提交于
Fail recovery if filter provides more records than original and corresponding unit-test, fix naming conventions
-
由 sdong 提交于
Summary: IO Posix depends on too many .h files. Move most of them to .cc files. Test Plan: make all Reviewers: anthony, rven, IslamAbdelRahman, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49479
-
由 Alex Loukissas 提交于
-
由 Alex Loukissas 提交于
-
由 Alex Loukissas 提交于
-
由 Siying Dong 提交于
Add Options.recycle_log_file_num for Recycling WAL Files
-
由 Islam AbdelRahman 提交于
Summary: Update rocksdb-lego-determinator to include running make check under ROCKSDB_LITE Test Plan: will be tested after landing in fbcode Reviewers: sdong, yhchiang, igor, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49065
-
- 25 10月, 2015 1 次提交
-
-
由 Vasili Svirski 提交于
-
- 23 10月, 2015 4 次提交
-
-
由 sdong 提交于
Summary: include/posix/io_posix.h doesn't not prevent multiple includes. Need to fix it. It is also breaking unity build. Test Plan: Run unity build and see error go away. Reviewers: rven, igor, IslamAbdelRahman, kradhakrishnan, anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49281
-
由 Igor Canadi 提交于
No need to #ifdef test only code on windows
-
由 Dmitri Smirnov 提交于
-
由 Siying Dong 提交于
Split posix storage backend into Env and library
-
- 22 10月, 2015 2 次提交
-
-
由 Javier González 提交于
Summary: This patch splits the posix storage backend into Env and the actual *File implementations. The motivation is to allow other Envs to use posix as a library. This enables a storage backend different from posix to split its secondary storage between a normal file system partition managed by posix, and it own media. Test Plan: No new functionality is added to posix Env or the library, thus the current tests should suffice.
-
由 Alexey Maykov 提交于
Summary: Need to pass through the memtable parameter. Test Plan: built, tested through myrocks Reviewers: igor, sdong, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49167
-
- 21 10月, 2015 6 次提交
-
-
由 Shusen Liu 提交于
Summary: in tools/db_crashtest.py, cmd_params['db'] by default is a lambda expression, not the actual db_name. fix by get the db_name before passing it to gen_cmd. Test Plan: run `make crashtest` Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49119
-
由 Vasili Svirski 提交于
-
由 Yueh-Hsuan Chiang 提交于
Summary: Use DEBUG_LEVEL=0 in make release and make clean Test Plan: make clean make release -j32 Reviewers: MarkCallaghan, sdong, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49125
-
由 Siying Dong 提交于
Do not build test only code and unit tests in Release builds
-
由 Dmitri Smirnov 提交于
Test code errors are currently blocking Windows Release builew We do not want spend time building in Release what we can not run We want to eliminate a source of most frequent errors when people check-in test only code which can not be built in Release. This feature will work only if you invoke msbuild against rocksdb.sln Invoking it against ALL_BUILD target will attempt to build everything.
-
由 Vasili Svirski 提交于
* add validation is git found * add validation is .git folder exists in project (project zip archive download without .git folder) * get head commit SHA if git found and .git folder exists Tested: * configure project by CMake 3.0.0 successfully (with and without git), with project zip archive (without .git folder) and with project cloned from github * configure project by command: cmake -G "Visual Studio 12 Win64" * build solution by Visual Studio * manually validate that file utils/build_version.cc contains valid head revision value
-