• H
    [Cherry-pick] Support memory eager deletion on recurrent OP (#19411) · cb74dac3
    Huihuang Zheng 提交于
    * Support memory eager deletion on recurrent OP (#17710)
    
    Test PaddingRNN on V100 GPU device.
    
    Test configuration: large model, padding mode (which is the mode using recurrentOp), one GPU.
                       
    GPU memory (MiB):   6414 (this PR)     vs   6837 (without this PR)
    Speed (steps/s):         10.28 (this PR)    vs    9.89 (without this PR)
    
    * Fix random test_recurrent_op failure (#18718)
    
    The change includes 3 things:
    
    1. Set CPU_NUM to 1 in the tests because the ParallelExecutor will print warning that CPU_NUM is not set and use default 1.
    
    2. Old tests compare two RNNs, hand written simple RNN and same RNN built by Paddle, but initialized RNN weights in numpy random and Paddle random separately. Fixed it by setting weights and bias values.
    
    3. Also set numpy random seed in the tests. Now the two RNNs diff can be smaller (rtol from 0.1, 0.2 to. 0.01) in the tests.
    cb74dac3
recurrent_op_helper.cc 9.6 KB