• L
    Make it possible to force the garbage collection of the oldest blob files (#8994) · 3e1bf771
    Levi Tamasi 提交于
    Summary:
    The current BlobDB garbage collection logic works by relocating the valid
    blobs from the oldest blob files as they are encountered during compaction,
    and cleaning up blob files once they contain nothing but garbage. However,
    with sufficiently skewed workloads, it is theoretically possible to end up in a
    situation when few or no compactions get scheduled for the SST files that contain
    references to the oldest blob files, which can lead to increased space amp due
    to the lack of GC.
    
    In order to efficiently handle such workloads, the patch adds a new BlobDB
    configuration option called `blob_garbage_collection_force_threshold`,
    which signals to BlobDB to schedule targeted compactions for the SST files
    that keep alive the oldest batch of blob files if the overall ratio of garbage in
    the given blob files meets the threshold *and* all the given blob files are
    eligible for GC based on `blob_garbage_collection_age_cutoff`. (For example,
    if the new option is set to 0.9, targeted compactions will get scheduled if the
    sum of garbage bytes meets or exceeds 90% of the sum of total bytes in the
    oldest blob files, assuming all affected blob files are below the age-based cutoff.)
    The net result of these targeted compactions is that the valid blobs in the oldest
    blob files are relocated and the oldest blob files themselves cleaned up (since
    *all* SST files that rely on them get compacted away).
    
    These targeted compactions are similar to periodic compactions in the sense
    that they force certain SST files that otherwise would not get picked up to undergo
    compaction and also in the sense that instead of merging files from multiple levels,
    they target a single file. (Note: such compactions might still include neighboring files
    from the same level due to the need of having a "clean cut" boundary but they never
    include any files from any other level.)
    
    This functionality is currently only supported with the leveled compaction style
    and is inactive by default (since the default value is set to 1.0, i.e. 100%).
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/8994
    
    Test Plan: Ran `make check` and tested using `db_bench` and the stress/crash tests.
    
    Reviewed By: riversand963
    
    Differential Revision: D31489850
    
    Pulled By: ltamasi
    
    fbshipit-source-id: 44057d511726a0e2a03c5d9313d7511b3f0c4eab
    3e1bf771
c.cc 179.7 KB