• A
    Bound L0->Lbase fanout in dynamic leveled compaction (#7325) · 177f8bd0
    Andrew Kryczka 提交于
    Summary:
    L0 score is based on size target and number of files. The size target
    used is `max_bytes_for_level_base`. However, the base level's size can
    dynamically expand in write burst mode. In fact, it can expand so much
    that L0->Lbase becomes the highest fanout in target sizes. This doesn't
    make sense from an efficiency perspective, so this PR bounds the
    L0->Lbase fanout to the smoothed level multiplier. The L0 scoring based
    on file count remains unchanged.
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/7325
    
    Test Plan:
    contrived benchmark that exhibits the problem:
    
    ```
    $ TEST_TMPDIR=/data/users/andrewkr/ ./db_bench -benchmarks=filluniquerandom,readrandom -write_buffer_size=1048576 -target_file_size_base=1048576 -max_bytes_for_level_base=4194304 -level0_file_num_compaction_trigger=4 -level_compaction_dynamic_level_bytes=true -compression_type=none -max_background_jobs=12 -rate_limiter_bytes_per_sec=104857600 -benchmark_write_rate_limit=10485760 -num=100000000
    ```
    
    Results:
    
    - "Burst W-Amp" is the write-amp near the end of the fillrandom benchmark
    - "Total W-Amp" is the write-amp after readrandom has run a while and all levels no longer need compaction
    
    Branch | Burst W-Amp | Total W-Amp | fillrandom (MB/s)
    -- | -- | -- | --
    master | 20.2 | 21.5 | 4.7
    dynamic-l0-score | 12.6 | 14.1 | 7.2
    
    Reviewed By: siying
    
    Differential Revision: D23412935
    
    Pulled By: ajkr
    
    fbshipit-source-id: f91f2067188e432dd39deab02f1c56f195057a0e
    177f8bd0
version_set.cc 234.1 KB