• P
    Fix block checksum for >=4GB, refactor (#6978) · 25a0d0ca
    Peter Dillinger 提交于
    Summary:
    Although RocksDB falls over in various other ways with KVs
    around 4GB or more, this change fixes how XXH32 and XXH64 were being
    called by the block checksum code to support >= 4GB in case that should
    ever happen, or the code copied for other uses.
    
    This change is not a schema compatibility issue because the checksum
    verification code would checksum the first (block_size + 1) mod 2^32
    bytes while the checksum construction code would checksum the first
    block_size mod 2^32 plus the compression type byte, meaning the
    XXH32/64 checksums for >=4GB block would not match about 255/256 times.
    
    While touching this code, I refactored to consolidate redundant
    implementations, improving diagnostics and performance tracking in some
    cases. Also used less confusing language in those diagnostics.
    
    Makes https://github.com/facebook/rocksdb/issues/6875 obsolete.
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/6978
    
    Test Plan:
    I was able to write a test for this using an SST file writer
    and VerifyChecksum in a reader. The test fails before the fix, though
    I'm leaving the test disabled because I don't think it's worth the
    expense of running regularly.
    
    Reviewed By: gg814
    
    Differential Revision: D22143260
    
    Pulled By: pdillinger
    
    fbshipit-source-id: 982993d16134e8c50bea2269047f901c1783726e
    25a0d0ca
block_based_table_reader.cc 131.4 KB