test=develop

Based on the comment here https://github.com/PaddlePaddle/Paddle/blob/b5f8784cab94eae785659787fc529870c87b254c/paddle/fluid/framework/details/build_strategy.h#L49

The unit test which compares Reduce and AllReduce must have diff. The PR_CI_Night runs on P40 machine and it has 8GB GPU, which is smaller than the 16GB normal CI machines. So we decrease the batch size in the past to make it runnable: https://github.com/PaddlePaddle/Paddle/pull/24651/files  . Decreasing the batch size makes the difference occurs often. So this PR replace the absolute delta by relative delta.

Before this PR, the unit test failure happens with probability about 1/100. After this PR it doesn't happen.

* fix the double grad bug for the star gan. test=develop

* update the retain_graph parameter doc. test=develop

* add the unit test for the retain_graph parameter. test=develop

* fix jit.save input type change error

* add unittes

* Refine Model

1. Take the network (instance of Layer) as the input of Model.
2. Refine set_dict/load_dict of Layer.
3. Refine Input interface, so update code sample about Input

* add fleet distributed metrics
* test=develop

* refine squeeze, test=develop

* update squeeze, test=develop

* refine compile-time infershape, test=develop

* add more unittest, test=develop

* follow comments, test=develop

* add update_api, test=develop

* follow comments, test=develop

* fix gen nccl id bug

* New features, add sinh and cosh op, test=develop

* remove duplicate test function and remove out paramters, test=develop

* Add out paramters temporary, remove later. test=develop

* remove out args, PR 25570, test=develop

* remove TestParameter, test=developx

* add test api for static dygraph, test=develop

* add backword unittests for sinh and cosh, test=develop

add fp64 support in sequence_pool, test=develop

* add more vision transfrom apis

* fix best_fit_allocator_test on windows, test=develop

* enable best_fit_allocator_test and test_math_op_patch_var_base, test=develop

* fix dataset py3 error
* test=develop

* add ci check for changing op-related api without core.ops, test=develop

* generate api_source_md5 file when build, test=develop

* add failed example, test=develop

* add failed example, test=develop

* handle exception, test=develop

* add op multiply, delete op elementwise_mul from math.py. test=develop,test=document_fix

* bug fix. test=develop,test=document_fix

* bug fix. test=develop,test=document_fix

* bug fix. test=develop,test=document_fix

* bug fix. test=develop,test=document_fix

* add unittest for multiply op. test=develop.

* fix code style. test=develop

* use random input. test=develop

* add test error case for static computation graph. test=develop

* add np.random.seed(7)

* increase input ndarray size. test=develop

* change float32 to float64. test=develop

Modify the print in Dy2stat for dynamic type. Unit test is covered in old test_print.py

* add reinforcement learning model test=develop

* align backward test=develop

* add gym in paddle_build.sh test=develop

* rm pip install in script test=develop

* refine paddle_build.sh test=develop

* fix sed error in macOS test=develop

* polish code test=develop

* fix scope problem

* refine code by reviewer comment

just skip case of distribute on windows

* update cross op parameters

windows CI scripts for xly

We found that the reason of multiple return error in SimNet is that I wrote wrong task_mode. If we set task_mode as "pairwise" correctly, which is a format of the model input data, the multiple return won't have problem in the unit test. In this PR we corrected the task_mode and enable multiple return in SimNet unit test.

refactor fleet api under paddle.fleet
update DistributedStrategy

* add reinforcement learning model test=develop

* align backward test=develop

* add gym in paddle_build.sh test=develop

* rm pip install in script test=develop

* refine paddle_build.sh test=develop

* fix sed error in macOS test=develop

* polish code test=develop

* refine infer ir pass tests, test=develop

* commit to change copy_from_cpu input to np array, test=develop

Enable multiple "return" statement in MNIST as Unit test in dy2stat.

* fix test_activation_op, test=develop

* remove paddle.mul related unittest, test=develop

* delete mul op. test=develop,test=document_fix

* bug fix. test=develop,test=document_fix

Add Support for SelectedRows for Transpose OP and Fix a Bug That SelectedRows Cannot be Supported in SimNet (#25536)

This PR fixes a bug that SelectedRows cannot be supported in SimNet. The reason of this bug is that dygraph basic_engine didn't copy var's type when the var needs to be accumulated during backward. So when a var is SelectedRows and needs to be accumulated, like SimNet which calls net for two times, the var's type will be changed to default LoDTensor thus bug happens. To fix it, we just also copy the type.

Without this PR, the accumulated SelectedRows parameters in dygraph will be changed into LoDTensor. So when we fixed the bug of supporting SelectedRows in SimNet, we found `test_imperative_lod_tensor_to_selected_rows` failed and threw the error that SelectedRows was not supported for Transpose OP. To fix it, too, this PR also added support for SelectedRows for Transpose OP.