• P
    Fix a major performance bug in 7.0 re: filter compatibility (#9736) · 91687d70
    Peter Dillinger 提交于
    Summary:
    Bloom filters generated by pre-7.0 releases are not read by
    7.0.x releases (and vice-versa) due to changes to FilterPolicy::Name()
    in https://github.com/facebook/rocksdb/issues/9590. This can severely impact read performance and read I/O on
    upgrade or downgrade with existing DB, but not data correctness.
    
    To fix, we go back using the old, unified name in SST metadata but (for
    a while anyway) recognize the aliases that could be generated by early
    7.0.x releases. This unfortunately requires a public API change to avoid
    interfering with all the good changes from https://github.com/facebook/rocksdb/issues/9590, but the API change
    only affects users with custom FilterPolicy, which should be very few.
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/9736
    
    Test Plan:
    manual
    
    Generate DBs with
    ```
    ./db_bench.7.0 -db=/dev/shm/rocksdb.7.0 -bloom_bits=10 -cache_index_and_filter_blocks=1 -benchmarks=fillrandom -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=0
    ```
    and similar. Compare with
    ```
    for IMPL in 6.29 7.0 fixed; do for DB in 6.29 7.0 fixed; do echo "Testing $IMPL on $DB:"; ./db_bench.$IMPL -db=/dev/shm/rocksdb.$DB -use_existing_db -readonly -bloom_bits=10 -benchmarks=readrandom -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=0 -duration=10 2>&1 | grep micros/op; done; done
    ```
    
    Results:
    ```
    Testing 6.29 on 6.29:
    readrandom   :      34.381 micros/op 29085 ops/sec;    3.2 MB/s (291999 of 291999 found)
    Testing 6.29 on 7.0:
    readrandom   :     190.443 micros/op 5249 ops/sec;    0.6 MB/s (52999 of 52999 found)
    Testing 6.29 on fixed:
    readrandom   :      40.148 micros/op 24907 ops/sec;    2.8 MB/s (249999 of 249999 found)
    Testing 7.0 on 6.29:
    readrandom   :     229.430 micros/op 4357 ops/sec;    0.5 MB/s (43999 of 43999 found)
    Testing 7.0 on 7.0:
    readrandom   :      33.348 micros/op 29986 ops/sec;    3.3 MB/s (299999 of 299999 found)
    Testing 7.0 on fixed:
    readrandom   :     152.734 micros/op 6546 ops/sec;    0.7 MB/s (65999 of 65999 found)
    Testing fixed on 6.29:
    readrandom   :      32.024 micros/op 31224 ops/sec;    3.5 MB/s (312999 of 312999 found)
    Testing fixed on 7.0:
    readrandom   :      33.990 micros/op 29390 ops/sec;    3.3 MB/s (294999 of 294999 found)
    Testing fixed on fixed:
    readrandom   :      28.714 micros/op 34825 ops/sec;    3.9 MB/s (348999 of 348999 found)
    ```
    
    Just paying attention to order of magnitude of ops/sec (short test
    durations, lots of noise), it's clear that with the fix we can read <= 6.29
    & >= 7.0 at full speed, where neither 6.29 nor 7.0 can on both. And 6.29
    release can properly read fixed DB at full speed.
    
    Reviewed By: siying, ajkr
    
    Differential Revision: D35057844
    
    Pulled By: pdillinger
    
    fbshipit-source-id: a46893a6af4bf084375ebe4728066d00eb08f050
    91687d70
c.cc 176.5 KB