• Z
    RocksDB Trace Analyzer (#4091) · 999d955e
    Zhichao Cao 提交于
    Summary:
    A framework of trace analyzing for RocksDB
    
    After collecting the trace by using the tool of [PR #3837](https://github.com/facebook/rocksdb/pull/3837). User can use the Trace Analyzer to interpret, analyze, and characterize the collected workload.
    **Input:**
    1. trace file
    2. Whole keys space file
    
    **Statistics:**
    1. Access count of each operation (Get, Put, Delete, SingleDelete, DeleteRange, Merge) in each column family.
    2. Key hotness (access count) of each one
    3. Key space separation based on given prefix
    4. Key size distribution
    5. Value size distribution if appliable
    6. Top K accessed keys
    7. QPS statistics including the average QPS and peak QPS
    8. Top K accessed prefix
    9. The query correlation analyzing, output the number of X after Y and the corresponding average time
        intervals
    
    **Output:**
    1. key access heat map (either in the accessed key space or whole key space)
    2. trace sequence file (interpret the raw trace file to line base text file for future use)
    3. Time serial (The key space ID and its access time)
    4. Key access count distritbution
    5. Key size distribution
    6. Value size distribution (in each intervals)
    7. whole key space separation by the prefix
    8. Accessed key space separation by the prefix
    9. QPS of each operation and each column family
    10. Top K QPS and their accessed prefix range
    
    **Test:**
    1. Added the unit test of analyzing Get, Put, Delete, SingleDelete, DeleteRange, Merge
    2. Generated the trace and analyze the trace
    
    **Implemented but not tested (due to the limitation of trace_replay):**
    1. Analyzing Iterator, supporting Seek() and SeekForPrev() analyzing
    2. Analyzing the number of Key found by Get
    
    **Future Work:**
    1.  Support execution time analyzing of each requests
    2.  Support cache hit situation and block read situation of Get
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/4091
    
    Differential Revision: D9256157
    
    Pulled By: zhichao-cao
    
    fbshipit-source-id: f0ceacb7eedbc43a3eee6e85b76087d7832a8fe6
    999d955e
Makefile 65.4 KB