test=develop

backward of backward: leaky_relu

test=develop

Speedup roi_perspective_transform op by caching the information of linear interpolation in forward (#17090)

* Cache the information of linear interpolation in forward and use it in backward.
test=develop

* Fix cuda kernel.
test=develop

test=develop

* Init mixed precision training interface

* Add fp16 test script

test=develop

* All initializers support float16

test=develop

* Code cleanup & add more code annotations

test=develop

* Update API spec

test=develop

* Add usage example in doc

test=develop

implement dygraph.parallel.DataParallel to hook reduce op.

* implement distributed transpiler with fleet

add checkpoint functions for graph. test=develop

* resolve #17057

Fixed the bug that fuse_relu/fuse_residual option couldn't be passed to class TestConv2dInt8Op.

test=develop

* Fix the bug of test_conv2d_int8_mkldnn case which raised by improper parameter passing.

test=develop

fix runtime_context_cache bug when gpu model has an op runs only on cpu

1. specify the cuda arch name and bin to decrease the compile time for inference test=develop
2. simplify the script and add comments
3. remove the fluid process from cicheck

update pip version in Readme to 1.4.1

test=develop

test=develop

update Readme and releasenote for 1.4.1

* Add fuse momenutum ops

fix nccl wrapper on windows

test=develop

* make_conv_cudnn_ws_size_configurable, test=develop

* change std::max to std::min
test=develop

fix potential hung in generate proposals, test=develop

fix split for dimension judgment

update ngraph version

test=develop

test=develop

reduce unittest time by rename testcuda to has_cuda

Support backward of backward for Relu and add a new gradient checker by comparing theoretical and numerical Jacobian. (#16862)

* Support backward of backward and a new gradient checker
* Rename decorators.py to decorator_helper.py, since Python on Windows CI has decorators package.

1. Add ReluDoubleGradMaker when register relu_grad.
2. Add a new gradient checker by comparing theoretical and numerical Jacobian.  Check double gradients by double_grad_check.

fix bug in save, test=develop

* move gc test to op_test
test=develop

* Revert "move gc test to op_test"

This reverts commit cf15da65.

* enable gc test in some ops
test=develop

enhance generate proposal labels, test=develop

* fix ramdom fail
test=develop

* fix cmakelist detecting problems

test=develop

* add doc for memory_optimize, test=develop

* update doc, test=develop

* doc update, test=develop