Dry run tests will skip `Op.Run` and just perform job scheduling. It helps to analysis dead lock in PE.

test=develop

test=develop

test=develop

test=develop

* add fp16 backward support
test=develop

* add sum_op fp16 test

* disable test_dist_save_load
test=develop

* add check_grad for sum

* add unit test for softmax_grad fp16
test=develop

* add scale_op unit test

* add mul_grad_op unit test for fp16

* add cross_entropy_grad and eman_grad unit test for fp16
test=develop

* fix cross_entropy unit test

* add pool2d fp16 unit test

* refine conv2d fp16 unit test
test=develop

* refine activation unit test
test=develop

* fix ci
test=develop

* follow zhihong's comment, copy from https://github.com/PaddlePaddle/Paddle/pull/12796
test=develop

test=develop

test=develop

test=develop

This reverts commit ce7d9b07.

* exhaustive search for cuDNN conv.
* Refine code and add unit testing.
* Clean code
* Fix model load in fluid/inference and unit testing in conv2d
* Follow comments.

test=develop

test=develop

test=develop

test=develop

test=develop

* add dam test

* update fuse_statis

* use separated dam model.

* Revert "use separated dam model."

This reverts commit 13e775c8.

* test=develop

* modify the cmake file about infer test, test=develop.

* remove one comment, test=develop.

test=develop

Warning only at first when CUDA CC not matched.

test=develop

* Fix build error of affine grid op in mac os.
test=develop

* Make function return reference.
test=develop

* wip

* add ref_by_trainer_id op

* ready to test

* fix ref inputs

* refine rpc_op_handle

* fix merge bug

test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

* enable pyreader use pin memory

* add py reader pin memory test test=develop

* Add affine grid generator.

* fix ffine grid.

* Add unitest.

* Add CPU kernel and fix unitest.

* Fix CPU kernel.

* Refine code.
test=develop

* Fix python api.
test=develop

* Update python api.
test=develop

* Fix comment.
test=develop

* Rename affine_grid_generator to affine_grid and enhence unitest.
test=develop

* Fix unitest.
test=develop

* fix dim0 in _load_slice_up_vars

* fix dim0 in _load_slice_up_vars, fix innershape in delete_var_op

* Revert "fix lookuptable in reduce strategy"

This reverts commit 0e722c5e

* add unit test for dist

* add unit test for dist, test=develop

* cancel revert, test=develop