• L
    Introduce a blob file reader class (#7461) · 22655a39
    Levi Tamasi 提交于
    Summary:
    The patch adds a class called `BlobFileReader` that can be used to retrieve blobs
    using the information available in blob references (e.g. blob file number, offset, and
    size). This will come in handy when implementing blob support for `Get`, `MultiGet`,
    and iterators, and also for compaction/garbage collection.
    
    When a `BlobFileReader` object is created (using the factory method `Create`),
    it first checks whether the specified file is potentially valid by comparing the file
    size against the combined size of the blob file header and footer (files smaller than
    the threshold are considered malformed). Then, it opens the file, and reads and verifies
    the header and footer. The verification involves magic number/CRC checks
    as well as checking for unexpected header/footer fields, e.g. incorrect column family ID
    or TTL blob files.
    
    Blobs can be retrieved using `GetBlob`. `GetBlob` validates the offset and compression
    type passed by the caller (because of the presence of the header and footer, the
    specified offset cannot be too close to the start/end of the file; also, the compression type
    has to match the one in the blob file header), and retrieves and potentially verifies and
    uncompresses the blob. In particular, when `ReadOptions::verify_checksums` is set,
    `BlobFileReader` reads the blob record header as well (as opposed to just the blob itself)
    and verifies the key/value size, the key itself, as well as the CRC of the blob record header
    and the key/value pair.
    
    In addition, the patch exposes the compression type from `BlobIndex` (both using an
    accessor and via `DebugString`), and adds a blob file read latency histogram to
    `InternalStats` that can be used with `BlobFileReader`.
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/7461
    
    Test Plan: `make check`
    
    Reviewed By: riversand963
    
    Differential Revision: D23999219
    
    Pulled By: ltamasi
    
    fbshipit-source-id: deb6b1160d251258b308d5156e2ec063c3e12e5e
    22655a39
Makefile 84.9 KB