• A
    Shard JemallocNodumpAllocator (#11400) · 925d8252
    Andrew Kryczka 提交于
    Summary:
    RocksDB's jemalloc no-dump allocator (`NewJemallocNodumpAllocator()`) was using a single manual arena. This arena's lock contention could be very high when thread caching is disabled for RocksDB blocks (e.g., when using `MALLOC_CONF='tcache_max:4096'` and `rocksdb_block_size=16384`).
    
    This PR changes the jemalloc no-dump allocator to use a configurable number of manual arenas. That number is required to be a power of two so we can avoid division. The allocator shards allocation requests randomly across those manual arenas.
    
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/11400
    
    Test Plan:
    - mysqld setup
      - Branch: fb-mysql-8.0.28 (https://github.com/facebook/mysql-5.6/commit/653eba2e56cfba4eac0c851ac9a70b2da9607527)
      - Build: `mysqlbuild.sh --clean --release`
      - Set env var `MALLOC_CONF='tcache_max:$tcache_max'`
      - Added CLI args `--rocksdb_cache_dump=false --rocksdb_block_cache_size=4294967296 --rocksdb_block_size=16384`
      - Ran under /usr/bin/time
    - Large database scenario
      - Setup command: `mysqlslap -h 127.0.0.1 -P 13020 --auto-generate-sql=1 --auto-generate-sql-load-type=write --auto-generate-sql-guid-primary=1 --number-char-cols=8 --auto-generate-sql-execute-number=262144 --concurrency=32 --no-drop`
      - Benchmark command: `mysqlslap -h 127.0.0.1 -P 13020 --query='select count(*) from mysqlslap.t1;' --number-of-queries=320 --concurrency=32`
      - Results:
    
    | tcache_max | num_arenas | Peak RSS MB (% change) | Query latency seconds (% change) |
    |---|---|---|---|
    | 4096 | **(baseline)** | 4541 | 37.1 |
    | 4096 | 1 | 4535 (-0.1%) | 36.7 (-1%) |
    | 4096 | 8 | 4687 (+3%) | 10.2 (-73%) |
    | 16384 | **(baseline)** | 4514 | 8.4 |
    | 16384 | 1 | 4526 (+0.3%) | 8.5 (+1%) |
    | 16384 | 8 | 4580 (+1%) | 8.5 (+1%) |
    
    Reviewed By: pdillinger
    
    Differential Revision: D45220794
    
    Pulled By: ajkr
    
    fbshipit-source-id: 9a50c9872bdef5d299e52b115a65ee8a5557d58d
    925d8252
可在Tags中查看这些版本中当前仓库的状态.
HISTORY.md 278.7 KB