• T
    Add DEBUG mode to the explain_memory_verbosity GUC · 825ca1e3
    Taylor Vesely 提交于
    The memory accounting system generates a new memory account for every
    execution node initialized in ExecInitNode. The address to these memory
    accounts is stored in the shortLivingMemoryAccountArray. If the memory
    allocated for shortLivingMemoryAccountArray is full, we will repalloc
    the array with double the number of available entries.
    
    After creating approximately 67000000 memory accounts, it will need to
    allocate more than 1GB of memory to increase the array size, and throw
    an ERROR, canceling the running query.
    
    PL/pgSQL and SQL functions will create new executors/plan nodes that
    must be tracked my the memory accounting system. This level of detail is
    not necessary for tracking memory leaks, and creating a separate memory
    account for every executor will use large amount of memory just to track
    these memory accounts.
    
    Instead of tracking millions of individual memory accounts, we
    consolidate any child executor account into a special 'X_NestedExecutor'
    account. If explain_memory_verbosity is set to 'detailed' and below,
    consolidate all child executors into this account.
    
    If more detail is needed for debugging, set explain_memory_verbosity to
    'debug', where, as was the previous behavior, every executor will be
    assigned its own MemoryAccountId.
    
    Originally we tried to remove nested execution accounts after they
    finish executing, but rolling over those accounts into a
    'X_NestedExecutor' account was impracticable to accomplish without the
    possibility of a future regression.
    
    If any accounts are created between nested executors that are not rolled
    over to an 'X_NestedExecutor' account, recording which accounts are
    rolled over can grow in the same way that the
    shortLivingMemoryAccountArray is growing today, and would also grow too
    large to reasonably fit in memory.
    
    If we were to iterate through the SharedHeaders every time that we
    finish a nested executor, it is not likely to be very performant.
    
    While we were at it, convert some of the convenience macros dealing with
    memory accounting for executor / planner node into functions, and move
    them out of memory accounting header files into the sole callers'
    compilation units.
    Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>
    Co-authored-by: NEkta Khanna <ekhanna@pivotal.io>
    Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
    Co-authored-by: NJoao Pereira <jdealmeidapereira@pivotal.io>
    Co-authored-by: NMelanie Plageman <mplageman@pivotal.io>
    825ca1e3
memaccounting_test.c 58.1 KB