1. 23 12月, 2016 1 次提交
    • A
      direct io write support · 972f96b3
      Aaron Gao 提交于
      Summary:
      rocksdb direct io support
      
      ```
      [gzh@dev11575.prn2 ~/rocksdb] ./db_bench -benchmarks=fillseq --num=1000000
      Initializing RocksDB Options from the specified file
      Initializing RocksDB Options from command-line flags
      RocksDB:    version 5.0
      Date:       Wed Nov 23 13:17:43 2016
      CPU:        40 * Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
      CPUCache:   25600 KB
      Keys:       16 bytes each
      Values:     100 bytes each (50 bytes after compression)
      Entries:    1000000
      Prefix:    0 bytes
      Keys per prefix:    0
      RawSize:    110.6 MB (estimated)
      FileSize:   62.9 MB (estimated)
      Write rate: 0 bytes/second
      Compression: Snappy
      Memtablerep: skip_list
      Perf Level: 1
      WARNING: Assertions are enabled; benchmarks unnecessarily slow
      ------------------------------------------------
      Initializing RocksDB Options from the specified file
      Initializing RocksDB Options from command-line flags
      DB path: [/tmp/rocksdbtest-112628/dbbench]
      fillseq      :       4.393 micros/op 227639 ops/sec;   25.2 MB/s
      
      [gzh@dev11575.prn2 ~/roc
      Closes https://github.com/facebook/rocksdb/pull/1564
      
      Differential Revision: D4241093
      
      Pulled By: lightmark
      
      fbshipit-source-id: 98c29e3
      972f96b3
  2. 10 2月, 2016 1 次提交
  3. 11 11月, 2015 2 次提交
    • D
      Make use of portable `uint64_t` type to make possible file access · 5270b33b
      Dmitri Smirnov 提交于
        in 64-bit.
      
        Currently, a signed off_t type is being used for the following
        interfaces for both offset and the length in bytes:
        * `Allocate`
        * `RangeSync`
      
        On Linux `off_t` is automatically either 32 or 64-bit depending on
        the platform. On Windows it is always a 32-bit signed long which
        limits file access and in particular space pre-allocation
        to effectively 2 Gb.
      
        Proposal is to replace off_t with uint64_t as a portable type
        always access files with 64-bit interfaces.
      
        May need to modify posix code but lack resources to test it.
      5270b33b
    • D
      Make use of portable `uint64_t` type to make possible file access · 5421c972
      Dmitri Smirnov 提交于
        in 64-bit.
      
        Currently, a signed off_t type is being used for the following
        interfaces for both offset and the length in bytes:
        * `Allocate`
        * `RangeSync`
      
        On Linux `off_t` is automatically either 32 or 64-bit depending on
        the platform. On Windows it is always a 32-bit signed long which
        limits file access and in particular space pre-allocation
        to effectively 2 Gb.
      
        Proposal is to replace off_t with uint64_t as a portable type
        always access files with 64-bit interfaces.
      
        May need to modify posix code but lack resources to test it.
      5421c972
  4. 28 10月, 2015 1 次提交
  5. 12 9月, 2015 1 次提交
    • D
      Refactor to support file_reader_writer on Windows. · 30e82d5c
      Dmitri Smirnov 提交于
        Summary. A change https://reviews.facebook.net/differential/diff/224721/
        Has attempted to move common functionality out of platform dependent
        code to a new facility called file_reader_writer.
        This includes:
        - perf counters
        - Buffering
        - RateLimiting
      
        However, the change did not attempt to refactor Windows code.
        To mitigate, we introduce new quering interfaces such as UseOSBuffer(),
        GetRequiredBufferAlignment() and ReaderWriterForward()
        for pure forwarding where required.
        Introduce WritableFile got a new method Truncate(). This is to communicate
        to the file as to how much data it has on close.
         - When space is pre-allocated on Linux it is filled with zeros implicitly,
          no such thing exist on Windows so we must truncate file on close.
         - When operating in unbuffered mode the last page is filled with zeros but we still want to truncate.
      
         Previously, Close() would take care of it but now buffer management is shifted to the wrappers and the file has
         no idea about the file true size.
      
         This means that Close() on the wrapper level must always include
         Truncate() as well as wrapper __dtor should call Close() and
         against double Close().
         Move buffered/unbuffered write logic to the wrapper.
         Utilize Aligned buffer class.
         Adjust tests and implement Truncate() where necessary.
         Come up with reasonable defaults for new virtual interfaces.
         Forward calls for RandomAccessReadAhead class to avoid double
         buffering and locking (double locking in unbuffered mode on WIndows).
      30e82d5c
  6. 22 7月, 2015 1 次提交
    • S
      RangeSync not to sync last 1MB of the file · 3dbf4ba2
      Siying Dong 提交于
      Summary:
      From other ones' investigation:
      
      "sync_file_range() behavior highly depends on kernel version and filesystem.
      
      xfs does neighbor page flushing outside of the specified ranges. For example, sync_file_range(fd, 8192, 16384) does not only trigger flushing page #3 to #4, but also flushing many more dirty pages (i.e. up to page#16)... Ranges of the sync_file_range() should be far enough from write() offset (at least 1MB)."
      
      Test Plan: make all check
      
      Reviewers: igor, rven, kradhakrishnan, yhchiang, IslamAbdelRahman, anthony
      
      Reviewed By: anthony
      
      Subscribers: yoshinorim, MarkCallaghan, sumeet, domas, dhruba, leveldb, ljin
      
      Differential Revision: https://reviews.facebook.net/D15807
      3dbf4ba2