Created by: zhangting2020
PR说明:
When there is no need to transform data , need_prepare_data_
will be set to false so that PrepareData
could be skipped at the rest iterations of this Op's execution to save the elapsed time. Note:we do not support skipping PrepareData in while block, because the Op's input may be changed by subsequent Ops, which may cause an error.
当Op不需要进行data transform时,可以在之后的迭代中跳过PrepareData以节省时间。但是while block中,Op会被执行多次,在test_rnn_decoder_api.py单测中发现,while block中存在这样的情况:
A(x, y) -> B(z, out=y)
A Op第一次执行时没有发生data transform,但是当A Op执行完,B Op修改了A Op的输入y,y被放在了与A Op不同的设备上。导致A Op在while的第二个step,需要进行data transform。因此while block中是否能跳过PrepareData不易判断,还可能导致错误发生。暂时不支持对while block中的Op跳过PrepareData。
使用要点:训练中该特性若要生效,需要设置use_program_cache=True;预测默认会生效 性能影响:
- 对语言模型的性能提升情况如下(v100):
模型 | 优化前 | 优化后 | 性能提升 |
---|---|---|---|
paddingrnn_large_padding | 12.821 step/s | 12.926 step/s | 0.8% |
paddingrnn_large_static | 20.419 setp/s | 20.679 step/s | 1.3% |
paddingrnn_small_padding | 28.817 setp/s | 29.256 step/s | 1.5% |
paddingrnn_small_static | 91.899 setp/s | 94.845 step/s | 3.2% |
- 预测模型(6148)
模型 | 优化前 | 优化后 | 性能提升 |
---|---|---|---|
PyramidDNN | 0.431324 ms per sample | 0.420885 ms per sample | 2.4% |