* support forward hook for dygraph, test=develop

* add optest for forward_hook in dygraph, test=develop

* add optest, test=develop

* polish code, test=develop

* add sample code, test=develop

* rename forwrd_hook to forward_post_hook, test=develop

* fix the api description, test=develop

* fix api description, test=develop

* prune train program by fetch_list, test=develop

* add unittest for prune, test=develop

* fix pruned feed, test=develop

* support ParallelExecutor and feed prune, test=develop

* add comments, test=develop

* update unittest, test=develop

* update unittests, test=develop

* remove debug code, test=develop

* support cond in clone, test=develop

* support cond in prune, test=develop

* support multiple minimize, test=develop

* support cache, test=develop

* fix _copy_param_info_from, test=develop

* support python2 str, test=develop

* remove debug code, test=develop

* fix bug of caching CompiledProgram, test=develop

* fix multi_device issue, test=develop

* tmp

* support tuple in fetch_list and overriding use_prune, test=develop

* dont use nonlocal in python2, test=develop

* remove nonlocal, test=develop

* code clean, test=develop

* code clean, test=develop

* feed list, test=develop

* test adam, test=develop

* follow comments, test=develop

* reduce duplicate code, test=develop

* update comments, test=develop

* add op inout check macro, test=develop

* fix enforce_test, test=develop

This reverts commit 48144e40.

* Add unittest for transformer prediction in dygraph_to_static.

* fix bug in fill_constant api. 

* Make transpose support size 0. test=develop

* support tensor array create read write in dygraph
test=develop

* minor fix
test=develop

* support tensor_array_to_tensor
test=develop

* add while loop case and err msg
test=develop

* refine ut
test=develop

* add L1Loss

* support L1Loss, test=develop

* add test, test=develop

* fix batch, test=develop

* follow comments, test=develop

* fix stat var in hogwild worker
* test=develop

* use global conv cache; test=develop

* use singleton cache; test=develop

* fix format error; test=develop

* add cudnn helper header; test=develop

* fix header error; test=develop

* fix mac unitest; test=develop

* fix mac unitest; test=develop

* fix file format; test=develop

* fix include file error, test=develop

* remove kernel_configs_ in class ExecutionContext and kernel_configs_map_ in class OperatorWithKernel, test=develop

* fix test_elementwise_mul_op_dim, test=develop
Co-authored-by: Nphlrain <phliuhongyu@126.com>

* Delete is_test from activation operators
test=develop

* Revent unneeded changes
test=develop

* add inplace_abn_op. test=develop

test=develop

* refine the error message of tensor_array_read_write Op

1. Add data_layer_not_check because it is needed in dygraph_to_static where input can be variable size
2. Remove warnings in static analysis because python cannot do exact static analysis

* add attr support for fusion group and add support for fill_constant and scale Op

* add fleet pslib pull and push sparse op and push dense op
* test=develop

test=develop

test=develop

* Remove Read-Only Basic Type Variable in loop_vars

test=develop

* Put class support in loop for future thing

test=develop

* Modify based on reviewer's comment

test=develop

As nccl comm is not created by CUDADeviceContext, it should be destroyed by the creator as the best practice of RAII.