• P
    Improve / clean up meta block code & integrity (#9163) · 230660be
    Peter Dillinger 提交于
    Summary:
    * Checksums are now checked on meta blocks unless specifically
    suppressed or not applicable (e.g. plain table). (Was other way around.)
    This means a number of cases that were not checking checksums now are,
    including direct read TableProperties in Version::GetTableProperties
    (fixed in meta_blocks ReadTableProperties), reading any block from
    PersistentCache (fixed in BlockFetcher), read TableProperties in
    SstFileDumper (ldb/sst_dump/BackupEngine) before table reader open,
    maybe more.
    * For that to work, I moved the global_seqno+TableProperties checksum
    logic to the shared table/ code, because that is used by many utilies
    such as SstFileDumper.
    * Also for that to work, we have to know when we're dealing with a block
    that has a checksum (trailer), so added that capability to Footer based
    on magic number, and from there BlockFetcher.
    * Knowledge of trailer presence has also fixed a problem where other
    table formats were reading blocks including bytes for a non-existant
    trailer--and awkwardly kind-of not using them, e.g. no shared code
    checking checksums. (BlockFetcher compression type was populated
    incorrectly.) Now we only read what is needed.
    * Minimized code duplication and differing/incompatible/awkward
    abstractions in meta_blocks.{cc,h} (e.g. SeekTo in metaindex block
    without parsing block handle)
    * Moved some meta block handling code from table_properties*.*
    * Moved some code specific to block-based table from shared table/ code
    to BlockBasedTable class. The checksum stuff means we can't completely
    separate it, but things that don't need to be in shared table/ code
    should not be.
    * Use unique_ptr rather than raw ptr in more places. (Note: you can
    std::move from unique_ptr to shared_ptr.)
    
    Without enhancements to GetPropertiesOfAllTablesTest (see below),
    net reduction of roughly 100 lines of code.
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/9163
    
    Test Plan:
    existing tests and
    * Enhanced DBTablePropertiesTest.GetPropertiesOfAllTablesTest to verify that
    checksums are now checked on direct read of table properties by TableCache
    (new test would fail before this change)
    * Also enhanced DBTablePropertiesTest.GetPropertiesOfAllTablesTest to test
    putting table properties under old meta name
    * Also generally enhanced that same test to actually test what it was
    supposed to be testing already, by kicking things out of table cache when
    we don't want them there.
    
    Reviewed By: ajkr, mrambacher
    
    Differential Revision: D32514757
    
    Pulled By: pdillinger
    
    fbshipit-source-id: 507964b9311d186ae8d1131182290cbd97a99fa9
    230660be
Makefile 89.9 KB