* refine fleetrun.ps_launch

* update fleet run for multi device support

* ps_graph support ps-gpu

* fix heter save

* add heter save unittest

* fix unittest & simple code

* update fleetrun

* fix fleetrun

* fix launch barrier

* fix role maker

* add paddlecloud rolemaker unittest

* rename heter_worker_device_guard

* mig deformable_conv to deform_conv2d

test=develop

* add load_op_xpu for Baidu Kunlun, test=kunlun

* add is_compiled_with_xpu for unit test, test=kunlun

* add is_compiled_with_xpu for unit test, test=kunlun

* refine parameters order && doc

* update rmsprop doc

* refine adam/transpose/unstack/stride_slice

* fix bug && doc

* fix doc

* bug fix

* bug fix

* fix doc

* fix doc

* fix doc

* fix doc

* depercate old strided_slice

* update doc

* set default value for name

* update doc

Mutiply allows non-tensor data input

* add flattern weight of lstm

* Add double grad in Squeeze and Unsqueeze

* Add double grad in Squeeze and Unsqueeze

Add xpu kernel for KUNLUN core:

* accuracy op
* sign op
* scale op
* sum op

Add default atol in xpu unittest.

* replace config by kwargs

* change save path form dir to prefix

* fix failed unittests

* revert unittest name change

* polish en docs

* add more tests for coverage

* add matmul doublegrad op

* fix compile errors

* modify code according to review

* delete float16

* Reimplement paddle.static.nn.fc, mainly change the parameters and English documentation.

* Polish the documentation format.

* disable_fuse_all_reduce

* fix test

* fix ut

* fix pool3d bug, test=develop

* fix unitest, test=develop

* fix test and fix pool2d bug, test=develop

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* Add api of constant in paddle.nn.initializer

* reimplement paddle.nn.functional.sigmoid_focal_loss. test=develop

* fix reduction error message. test=develop

* fix exp. test=develop

* reset the shape of logit. test=develop

* delete disable_static in example. test=develop

* fix bilateral bug

Add Simnet Test for V2 APIs.

We used tool from https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0-beta/guides/migration_cn.html#paddle1-xpaddle2-0beta to do v1.x to v2 transformation.

This PR pulled changes from #27430, please DO NOT merge before #27430 is merged

* Add test skip from cmake

* Remove print

* Add avx instructions check

* Small fix

* Change function name

* Change uint to unsigned int

* test=develop, update example

* move dygraph amp api to paddle

* refine code and add unit test

* add multinomial cpu kernel

* fix C++ notype error

* fix windows ci array len error

* let array len be const

* change array to vector

* add cuda kernrl with num_distribution is 1, and not support replacement=False

* add multinomial python api

* support num_distribution different multinomial distributions

* add categorical class

* fix test_distribution enable_static error

* add unittest for different setting of Categorical

* optimize format

* little change

* little change

* add raise error if shape not match, optimize format

* fix windows CI dtype error in concat

* little changes

* little changes2

* change values type to int64

* change values type to int64

* change values type to int64

* Add test_resnet_v2.py test=develop

* refine grid_sample and temporal_shift

* fleet support non_distributed training in dygraph mode; test=develop