Skip to content

R
rocksdb

kvdb / rocksdb
12 个月 前同步成功

通知 0
Star 0
Fork 0
R
rocksdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 21 10月, 2015 1 次提交
    • D
      Do not build test only code and unit tests in Release builds · e154ee08
      由 Dmitri Smirnov 提交于 
        Test code errors are currently blocking Windows Release builew
  We do not want spend time building in Release what we can not run
  We want to eliminate a source of most frequent errors when people
  check-in test only code which can not be built in Release.
  This feature will work only if you invoke msbuild against rocksdb.sln
  Invoking it against ALL_BUILD target will attempt to build everything.
      e154ee08
  3. 18 10月, 2015 1 次提交
  5. 17 10月, 2015 1 次提交
  7. 15 10月, 2015 3 次提交
  9. 13 10月, 2015 1 次提交
  11. 07 10月, 2015 1 次提交
  13. 01 10月, 2015 1 次提交
  15. 30 9月, 2015 2 次提交
    • Y
      RocksDB Options file format and its serialization / deserialization. · 74b100ac
      由 Yueh-Hsuan Chiang 提交于 
      Summary:
This patch defines the format of RocksDB options file, which
follows the INI file format, and implements functions for its
serialization and deserialization.  An example RocksDB options
file can be found in examples/rocksdb_option_file_example.ini.

A typical RocksDB options file has three sections, which are
Version, DBOptions, and more than one CFOptions.  The RocksDB
options file in general follows the basic INI file format
with the following extensions / modifications:
 * Escaped characters
   We escaped the following characters:
    - \n -- line feed - new line
    - \r -- carriage return
    - \\ -- backslash \
    - \: -- colon symbol :
    - \# -- hash tag #
 * Comments
   We support # style comments.  Comments can appear at the ending
   part of a line.
 * Statements
   A statement is of the form option_name = value.
   Each statement contains a '=', where extra white-spaces
   are supported. However, we don't support multi-lined statement.
   Furthermore, each line can only contain at most one statement.
 * Section
   Sections are of the form [SecitonTitle "SectionArgument"],
   where section argument is optional.
 * List
   We use colon-separated string to represent a list.
   For instance, n1:n2:n3:n4 is a list containing four values.

Below is an example of a RocksDB options file:

[Version]
  rocksdb_version=4.0.0
  options_file_version=1.0
[DBOptions]
  max_open_files=12345
  max_background_flushes=301
[CFOptions "default"]
[CFOptions "the second column family"]
[CFOptions "the third column family"]

Test Plan: Added many tests in options_test.cc

Reviewers: igor, IslamAbdelRahman, sdong, anthony

Reviewed By: anthony

Subscribers: maykov, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D46059
      74b100ac
    • D
      Improve CI build and build switches · 2e7506d8
      由 Dmitri Smirnov 提交于 
        Add an optimized build config switch for faster test runs
  Change compiler options to introduce more opitmizations and be more inline with MS internal switches.
  Make appveyor build to utilize all the avaiable cores on the VM (parallel)
  Introduce new appveyor configuration for daily test runs as it would take too long
  to run db_test after each checkin even in paralell.
  With some exclusions we make it in 38 minutes. We currently fail to install ramdisk during the build.
  Add a powershell script to faicilitate paralell run for db_test cases.
      2e7506d8
  17. 24 9月, 2015 1 次提交
    • A
      Remove ldb HexToString method's usage of sscanf · 4805fa0e
      由 Assaf Sela 提交于 
      Summary:
Fix hex2String performance issues by removing sscanf dependency.
Also fixed some edge case handling (odd length, bad input).

Test Plan: Created a test file which called old and new implementation, and validated results are the same. I'll paste results in the phabricator diff.

Reviewers: igor, rven, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, sdong

Reviewed By: sdong

Subscribers: thatsafunnyname, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D46785
      4805fa0e
  19. 11 9月, 2015 1 次提交
    • A
      Refactored common code of Builder/CompactionJob out into a CompactionIterator · 8aa1f151
      由 Andres Noetzli 提交于 
      Summary:
Builder and CompactionJob share a lot of fairly complex code. This patch
refactors this code into a separate class, the CompactionIterator. Because the
shared code is fairly complex, this patch hopefully improves maintainability.
While there are is a lot of potential for further improvements, the patch is
intentionally pretty close to the original structure because the change is
already complex enough.

Test Plan: make clean all check && ./db_stress

Reviewers: rven, anthony, yhchiang, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D46197
      8aa1f151
  21. 09 9月, 2015 1 次提交
    • A
      TransactionDB Custom Locking API · 5e94f68f
      由 agiardullo 提交于 
      Summary:
Prototype of API to allow MyRocks to override default Mutex/CondVar used by transactions with their own implementations.  They would simply need to pass their own implementations of Mutex/CondVar to the templated TransactionDB::Open().

Default implementation of TransactionDBMutex/TransactionDBCondVar provided (but the code is not currently changed to use this).

Let me know if this API makes sense or if it should be changed

Test Plan: n/a

Reviewers: yhchiang, rven, igor, sdong, spetrunia

Reviewed By: spetrunia

Subscribers: maykov, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43761
      5e94f68f
  23. 01 9月, 2015 1 次提交
    • A
      Support static Status messages · 77a28615
      由 agiardullo 提交于 
      Summary: Provide a way to specify a detailed static error message for a Status without incurring a memcpy.  Let me know what people think of this approach.

Test Plan: added simple test

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D44259
      77a28615
  25. 26 8月, 2015 1 次提交
  27. 25 8月, 2015 1 次提交
    • A
      Common base class for transactions · 20d1e547
      由 agiardullo 提交于 
      Summary:
As I keep adding new features to transactions, I keep creating more duplicate code.  This diff cleans this up by creating a base implementation class for Transaction and OptimisticTransaction to inherit from.

The code in TransactionBase.h/.cc is all just copied from elsewhere.  The only entertaining part of this class worth looking at is the virtual TryLock method which allows OptimisticTransactions and Transactions to share the same common code for Put/Get/etc.

The rest of this diff is mostly red and easy on the eyes.

Test Plan: No functionality change.  existing tests pass.

Reviewers: sdong, jkedgar, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D45135
      20d1e547
  29. 12 8月, 2015 2 次提交
    • A
      Pessimistic Transactions · c2f2cb02
      由 agiardullo 提交于 
      Summary:
Initial implementation of Pessimistic Transactions.  This diff contains the api changes discussed in D38913.  This diff is pretty large, so let me know if people would prefer to meet up to discuss it.

MyRocks folks:  please take a look at the API in include/rocksdb/utilities/transaction[_db].h and let me know if you have any issues.

Also, you'll notice a couple of TODOs in the implementation of RollbackToSavePoint().  After chatting with Siying, I'm going to send out a separate diff for an alternate implementation of this feature that implements the rollback inside of WriteBatch/WriteBatchWithIndex.  We can then decide which route is preferable.

Next, I'm planning on doing some perf testing and then integrating this diff into MongoRocks for further testing.

Test Plan: Unit tests, db_bench parallel testing.

Reviewers: igor, rven, sdong, yhchiang, yoshinorim

Reviewed By: sdong

Subscribers: hermanlee4, maykov, spetrunia, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D40869
      c2f2cb02
    • A
      Fix Windows build by adding snapshot_impl to CMakeLists · 6b2d5703
      由 agiardullo 提交于 
      Test Plan: untested

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D44049
      6b2d5703
  31. 06 8月, 2015 2 次提交
    • S
      Add util/delete_scheduler_impl.cc to CMakeLists.txt · 48e6e9aa
      由 sdong 提交于 
      Summary: util/delete_scheduler_impl.cc is missing CMakeLists.txt, which fails Windows build. Add it.

Test Plan: Build in both of Windows and Linux

Reviewers: anthony, rven, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43641
      48e6e9aa
    • S
      Add two unit tests for SyncWAL() · 7ccd1c80
      由 sdong 提交于 
      Summary:
Add two unit tests for SyncWAL(). One makes sure SyncWAL() doesn't block writes in the other thread. Another one makes sure SyncWAL() doesn't wait ongoing writes to finish before being executed.

Create a new test file db_wal_test and move two WAL related tests from db_test to here.

Test Plan: Run the new tests

Reviewers: IslamAbdelRahman, rven, kradhakrishnan, kolmike, tnovak, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43605
      7ccd1c80
  33. 04 8月, 2015 1 次提交
    • Y
      Add CompactOnDeletionCollector in utilities/table_properties_collectors. · 26894303
      由 Yueh-Hsuan Chiang 提交于 
      Summary:
This diff adds CompactOnDeletionCollector in utilities/table_properties_collectors,
which applies a sliding window to a sst file and mark this file as need-compaction
when it observe enough deletion entries within the consecutive keys covered by
the sliding window.

Test Plan: compact_on_deletion_collector_test

Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yoshinorim, sdong

Reviewed By: sdong

Subscribers: maykov, dhruba

Differential Revision: https://reviews.facebook.net/D41175
      26894303
  35. 22 7月, 2015 3 次提交
    • D
      Add missing tests, fix db_sanity · 31b35c90
      由 Dmitri Smirnov 提交于 
       Add heap_test, merge_helper_test
 Fix uninitialized pointers in db_sanity_test that cause SIGSEV when DB::Open fails in case compression is not linked.
      31b35c90
    • D
      Fix windows build after refactoring · ac5e441a
      由 Dmitri Smirnov 提交于 
        Missing and duplicate files in CMake
  Missing definition of port::Crash
      ac5e441a
    • S
      RangeSync not to sync last 1MB of the file · 3dbf4ba2
      由 Siying Dong 提交于 
      Summary:
From other ones' investigation:

"sync_file_range() behavior highly depends on kernel version and filesystem.

xfs does neighbor page flushing outside of the specified ranges. For example, sync_file_range(fd, 8192, 16384) does not only trigger flushing page #3 to #4, but also flushing many more dirty pages (i.e. up to page#16)... Ranges of the sync_file_range() should be far enough from write() offset (at least 1MB)."

Test Plan: make all check

Reviewers: igor, rven, kradhakrishnan, yhchiang, IslamAbdelRahman, anthony

Reviewed By: anthony

Subscribers: yoshinorim, MarkCallaghan, sumeet, domas, dhruba, leveldb, ljin

Differential Revision: https://reviews.facebook.net/D15807
      3dbf4ba2
  37. 21 7月, 2015 3 次提交
  39. 17 7月, 2015 1 次提交
    • D
      Ensure Windows build w/o port/port.h in public headers · d1a45718
      由 Dmitri Smirnov 提交于 
       - Remove make file defines from public headers and use _WIN32 because it is compiler defined
 - use __GNUC__ and __clang__ to guard non-portable attributes
 - add #include "port/port.h" to some new .cc files.
 - minor changes in CMakeLists to reflect recent changes
      d1a45718
  41. 16 7月, 2015 2 次提交
    • D
      Ensure Windows build w/o port/port.h in public headers · 247690fe
      由 Dmitri Smirnov 提交于 
       - Remove make file defines from public headers and use _WIN32 because it is compiler defined
 - use __GNUC__ and __clang__ to guard non-portable attributes
 - add #include "port/port.h" to some new .cc files.
 - minor changes in CMakeLists to reflect recent changes
      247690fe
    • A
      move convenience.h out of utilities · 81d07262
      由 agiardullo 提交于 
      Summary: Moved convenience.h out of utilities to remove a dependency on utilities in db.

Test Plan: unit tests.  Also compiled a link to the old location to verify the _Pragma works.

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D42201
      81d07262
  43. 15 7月, 2015 5 次提交
  45. 10 7月, 2015 1 次提交
  47. 08 7月, 2015 2 次提交
  49. 02 7月, 2015 1 次提交
    • D
      Windows Port from Microsoft · 18285c1e
      由 Dmitri Smirnov 提交于 
       Summary: Make RocksDb build and run on Windows to be functionally
 complete and performant. All existing test cases run with no
 regressions. Performance numbers are in the pull-request.

 Test plan: make all of the existing unit tests pass, obtain perf numbers.

 Co-authored-by: Praveen Rao praveensinghrao@outlook.com
 Co-authored-by: Sherlock Huang baihan.huang@gmail.com
 Co-authored-by: Alex Zinoviev alexander.zinoviev@me.com
 Co-authored-by: Dmitri Smirnov dmitrism@microsoft.com
      18285c1e