test=develop

backward of backward: leaky_relu

Speedup roi_perspective_transform op by caching the information of linear interpolation in forward (#17090)

* Cache the information of linear interpolation in forward and use it in backward.
test=develop

* Fix cuda kernel.
test=develop

* Init mixed precision training interface

* Add fp16 test script

test=develop

* All initializers support float16

test=develop

* Code cleanup & add more code annotations

test=develop

* Update API spec

test=develop

* Add usage example in doc

test=develop

implement dygraph.parallel.DataParallel to hook reduce op.

* implement distributed transpiler with fleet

* resolve #17057

Fixed the bug that fuse_relu/fuse_residual option couldn't be passed to class TestConv2dInt8Op.

test=develop

* Fix the bug of test_conv2d_int8_mkldnn case which raised by improper parameter passing.

test=develop

* Add fuse momenutum ops

test=develop

Support backward of backward for Relu and add a new gradient checker by comparing theoretical and numerical Jacobian. (#16862)

* Support backward of backward and a new gradient checker
* Rename decorators.py to decorator_helper.py, since Python on Windows CI has decorators package.

1. Add ReluDoubleGradMaker when register relu_grad.
2. Add a new gradient checker by comparing theoretical and numerical Jacobian.  Check double gradients by double_grad_check.

* move gc test to op_test
test=develop

* Revert "move gc test to op_test"

This reverts commit cf15da65c38f57c91f53b3d8b3c2365d4aa86016.

* enable gc test in some ops
test=develop

* fix ramdom fail
test=develop

test=develop

* add parallel build script to ci test=develop
* 1. classify the test case as single card/two cards/multiple cards type
   2. run test case according to the run type

* Speed affine_channel_op unit testing
* Add check in tensor_py
* Fix ONLY_CPU Compiling

Update the filter generation mechanism that it could generate the negative parameter.
The original calling(np.random.random()) couldn't simulate the conv/relu fusion case.

test=develop

* accelerate test_ir_memory_optimize_nlp, test=develop

* accelerate test_ir_memory_optimize_nlp, test=develop

Rename the testcuda function to has_cuda, it will elimate the unnecessary testing.
test=develop

* speedup gc and inplace softmax_with_cross_entropy_grad
test=develop

* refine models gpu mem
Merge skip vars and warning messages of mem opt
remove relu mem opt
test=develop

* follow comments
test=develop

test=develop

* Support seq len equal to 0 in sequence ops

test=develop

* Add more test cases

* Fix some comments

test=develop

* Fix py3 error

test=develop

* fix sampling id op bug, test=develop

test=develop

test=develop

test = develop
resolve #16764

test=develop

test=develop

test=develop

* enable recurrent op test=develop

* make test_dyn_rnn faster

test = develop
resolve #16764