• P
    Add manifest fix-up utility for file temperatures (#9683) · a8a422e9
    Peter Dillinger 提交于
    Summary:
    The goal of this change is to allow changes to the "current" (in
    FileSystem) file temperatures to feed back into DB metadata, so that
    they can inform decisions and stats reporting. In part because of
    modular code factoring, it doesn't seem easy to do this automagically,
    where opening an SST file and observing current Temperature different
    from expected would trigger a change in metadata and DB manifest write
    (essentially giving the deep read path access to the write path). It is also
    difficult to do this while the DB is open because of the limitations of
    LogAndApply.
    
    This change allows updating file temperature metadata on a closed DB
    using an experimental utility function UpdateManifestForFilesState()
    or `ldb update_manifest --update_temperatures`. This should suffice for
    "migration" scenarios where outside tooling has placed or re-arranged DB
    files into a (different) tiered configuration without going through
    RocksDB itself (currently, only compaction can change temperature
    metadata).
    
    Some details:
    * Refactored and added unit test for `ldb unsafe_remove_sst_file` because
    of shared functionality
    * Pulled in autovector.h changes from https://github.com/facebook/rocksdb/issues/9546 to fix SuperVersionContext
    move constructor (related to an older draft of this change)
    
    Possible follow-up work:
    * Support updating manifest with file checksums, such as when a
    new checksum function is used and want existing DB metadata updated
    for it.
    * It's possible that for some repair scenarios, lighter weight than
    full repair, we might want to support UpdateManifestForFilesState() to
    modify critical file details like size or checksum using same
    algorithm. But let's make sure these are differentiated from modifying
    file details in ways that don't suspect corruption (or require extreme
    trust).
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/9683
    
    Test Plan: unit tests added
    
    Reviewed By: jay-zhuang
    
    Differential Revision: D34798828
    
    Pulled By: pdillinger
    
    fbshipit-source-id: cfd83e8fb10761d8c9e7f9c020d68c9106a95554
    a8a422e9
ldb_cmd_test.cc 37.8 KB