• D
    Reduce write amplification by merging files in L0 back into L0 · 554c06dd
    Dhruba Borthakur 提交于
    Summary:
    There is a new option called hybrid_mode which, when switched on,
    causes HBase style compactions.  Files from L0 are
    compacted back into L0. This meat of this compaction algorithm
    is in PickCompactionHybrid().
    
    All files reside in L0. That means all files have overlapping
    keys. Each file has a time-bound, i.e. each file contains a
    range of keys that were inserted around the same time. The
    start-seqno and the end-seqno refers to the timeframe when
    these keys were inserted.  Files that have contiguous seqno
    are compacted together into a larger file. All files are
    ordered from most recent to the oldest.
    
    The current compaction algorithm starts to look for
    candidate files starting from the most recent file. It continues to
    add more files to the same compaction run as long as the
    sum of the files chosen till now is smaller than the next
    candidate file size. This logic needs to be debated
    and validated.
    
    The above logic should reduce write amplification to a
    large extent... will publish numbers shortly.
    
    Test Plan: dbstress runs for 6 hours with no data corruption (tested so far).
    
    Differential Revision: https://reviews.facebook.net/D11289
    554c06dd
db_test.cc 97.4 KB