1. 06 3月, 2014 4 次提交
    • L
      output perf_context in db_bench readrandom · 04298f8c
      Lei Jin 提交于
      Summary:
      Add helper function to print perf context data in db_bench if enabled.
      I didn't find any code that actually exports perf context data. Not sure
      if I missed anything
      
      Test Plan: ran db_bench
      
      Reviewers: haobo, sdong, igor
      
      Reviewed By: igor
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16575
      04298f8c
    • L
      fix db_bench to use HashSkipList for real · 64138b5d
      Lei Jin 提交于
      Summary:
      For HashSkipList case, DBImpl has sanity check to see if prefix_extractor in
      options is the same as the one in memtable factory. If not, it falls
      back to SkipList. As result, I was experimenting with SkipList
      performance. No wonder it is much worse than LinkedList
      
      Test Plan: ran benchmark
      
      Reviewers: haobo, sdong, igor
      
      Reviewed By: igor
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16569
      64138b5d
    • L
      config max_background_flushes in db_bench · 51560ba7
      Lei Jin 提交于
      Summary: as title
      
      Test Plan: make release
      
      Reviewers: haobo, sdong, igor
      
      Reviewed By: haobo
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16437
      51560ba7
    • I
      MergingIterator assertion · c0ccf436
      Igor Canadi 提交于
      Summary: I wrote a test that triggers assertion in MergingIterator. I have not touched that code ever, so I'm looking for somebody with good understanding of the MergingIterator code to fix this. The solution is probably a one-liner. Let me know if you're willing to take a look.
      
      Test Plan: This test fails with an assertion `use_heap_ == false`
      
      Reviewers: dhruba, haobo, sdong, kailiu
      
      Reviewed By: sdong
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16521
      c0ccf436
  2. 05 3月, 2014 2 次提交
  3. 04 3月, 2014 2 次提交
    • K
      Add a hash-index component for block · 906f3dca
      kailiu 提交于
      Summary:
      this is the key component extracted from diff: https://reviews.facebook.net/D14271
      I separate it to a dedicated patch to make the review easier.
      
      Test Plan: added a unit test and passed it.
      
      Reviewers: haobo, sdong, dhruba
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16245
      906f3dca
    • I
      Fix a group commit bug in LogAndApply · 5142b370
      Igor Canadi 提交于
      Summary:
      EncodeTo(&record) does not overwrite, it appends to it.
      
      This means that group commit log and apply will look something like:
      record1
      record1record2
      record1record2record3
      
      I'm surprised this didn't show up in production, but I think the reason is that MANIFEST group commit almost never happens.
      
      This bug turned up in column family work, where opening a database failed with "adding a same column family twice".
      
      Test Plan: Tested the change in column family branch and observed that the problem is gone (with db_stress)
      
      Reviewers: dhruba, haobo
      
      Reviewed By: dhruba
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16461
      5142b370
  4. 01 3月, 2014 3 次提交
    • K
      Remove the terrible hack in for flush_block_policy_factory · bf86af51
      kailiu 提交于
      Summary:
      Previous code is too convoluted and I must be drunk for letting
      such code to be written without a second thought.
      
      Thanks to the discussion with @sdong, I added the `Options` when
      generating the flusher, thus avoiding the tricks.
      
      Just FYI: I resisted to add Options in flush_block_policy.h since I
      wanted to avoid cyclic dependencies: FlushBlockPolicy dpends on Options
      and Options also depends FlushBlockPolicy... While I appreciate my
      effort to prevent it, the old design turns out creating more troubles than
      it tried to avoid.
      
      Test Plan: ran ./table_test
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      CC: sdong, leveldb
      
      Differential Revision: https://reviews.facebook.net/D16503
      bf86af51
    • I
      Make Log::Reader more robust · 58ca641d
      Igor Canadi 提交于
      Summary:
      This diff does two things:
      (1) Log::Reader does not report a corruption when the last record in a log or manifest file is truncated (meaning that log writer died in the middle of the write). Inherited the code from LevelDB: https://code.google.com/p/leveldb/source/detail?r=269fc6ca9416129248db5ca57050cd5d39d177c8#
      (2) Turn off mmap writes for all writes to log and manifest files
      
      (2) is necessary because if we use mmap writes, the last record is not truncated, but is actually filled with zeros, making checksum fail. It is hard to recover from checksum failing.
      
      Test Plan:
      Added unit tests from LevelDB
      Actually recovered a "corrupted" MANIFEST file.
      
      Reviewers: dhruba, haobo
      
      Reviewed By: haobo
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16119
      58ca641d
    • Y
      Add ReadOptions to TransactionLogIterator. · a77527f2
      Yueh-Hsuan Chiang 提交于
      Summary:
      Add an optional input parameter ReadOptions to DB::GetUpdateSince(),
      which allows the verification of checksums to be disabled by setting
      ReadOptions::verify_checksums to false.
      
      Test Plan: Tests are done off-line and will not be included in the regular unit test.
      
      Reviewers: igor
      
      Reviewed By: igor
      
      CC: leveldb, xjin, dhruba
      
      Differential Revision: https://reviews.facebook.net/D16305
      a77527f2
  5. 28 2月, 2014 1 次提交
  6. 26 2月, 2014 3 次提交
    • I
      Schedule flush when waiting on flush · 42095163
      Igor Canadi 提交于
      Summary:
      This will also help with avoiding the deadlock. If a flush failed and we're waiting for a memtable to be flushed, we should schedule a new flush and hope a new one succeedes.
      
      If paranoid_checks = false, Wait() will still hang on ENOSPC, but at least it will automatically continue when the space frees up. Current behavior both hangs and deadlocks.
      
      Also, I renamed some 'compaction' to 'flush'. 'compaction' was leveldb way of saying things.
      
      Test Plan: make check
      
      Reviewers: dhruba, haobo, ljin
      
      Reviewed By: haobo
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16281
      42095163
    • L
      expose wal_dir in db_bench · dea894ef
      Lei Jin 提交于
      Summary: as title
      
      Test Plan: ran db_bench
      
      Reviewers: dhruba, haobo
      
      Reviewed By: haobo
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D16269
      dea894ef
    • A
      A few more C API functions. · 72aacf6b
      Albert Strasheim 提交于
      72aacf6b
  7. 25 2月, 2014 2 次提交
  8. 24 2月, 2014 2 次提交
  9. 20 2月, 2014 4 次提交
  10. 15 2月, 2014 1 次提交
  11. 14 2月, 2014 2 次提交
  12. 13 2月, 2014 5 次提交
  13. 12 2月, 2014 2 次提交
    • K
      Fix problem 3 for issue #80 · 265150cb
      kailiu 提交于
      265150cb
    • S
      Reduce malloc of iterators in Get() code paths · 33042669
      Siying Dong 提交于
      Summary:
      This patch optimized Get() code paths by avoiding malloc of iterators. Iterator creation is moved to mem table rep implementations, where a callback is called when any key is found. This is the same practice as what we do in (SST) table readers.
      
      db_bench result for readrandom following a writeseq, with no compression, single thread and tmpfs, we see throughput improved to 144958 from 139027, about 3%.
      
      Test Plan: make all check
      
      Reviewers: dhruba, haobo, igor
      
      Reviewed By: haobo
      
      CC: leveldb, yhchiang
      
      Differential Revision: https://reviews.facebook.net/D14685
      33042669
  14. 11 2月, 2014 1 次提交
  15. 09 2月, 2014 1 次提交
  16. 08 2月, 2014 2 次提交
    • K
      Make table properties shareable · 161ab42a
      kailiu 提交于
      Summary:
      We are going to expose properties of all tables to end users through "some" db interface.
      However, current design doesn't naturally fit for this need, which is because:
      
      1. If a table presents in table cache, we cannot simply return the reference to its table properties, because the table may be destroy after compaction (and we don't want to hold the ref of the version).
      2. Copy table properties is OK, but it's slow.
      
      Thus in this diff, I change the table reader's interface to return a shared pointer (for const table properties), instead a const refernce.
      
      Test Plan: `make check` passed
      
      Reviewers: haobo, sdong, dhruba
      
      Reviewed By: haobo
      
      CC: leveldb
      
      Differential Revision: https://reviews.facebook.net/D15999
      161ab42a
    • I
      Readrandom with tailing iterator · 1560bb91
      Igor Canadi 提交于
      Summary:
      Added an option for readrandom benchmark to run with tailing iterator instead of Get. Benefit of tailing iterator is that it doesn't require locking DB mutex on access.
      
      I also have some results when running on my machine. The results highly depend on number of cache shards. With our current benchmark setting of 4 table cache shards and 6 block cache shards, I don't see much improvements of using tailing iterator. In that case, we're probably seeing cache mutex contention.
      
      Here are the results for different number of shards
      
          cache shards       tailing iterator        get
             6                      1.38M           1.16M
            10                      1.58M           1.15M
      
      As soon as we get rid of cache mutex contention, we're seeing big improvements in using tailing iterator vs. ordinary get.
      
      Test Plan: ran regression test
      
      Reviewers: dhruba, haobo, ljin, kailiu, sding
      
      Reviewed By: haobo
      
      CC: tnovak
      
      Differential Revision: https://reviews.facebook.net/D15867
      1560bb91
  17. 04 2月, 2014 3 次提交