Add distribution supported

* fix the rnn mask memory bug for out of read

* update the code for the rnn

* softmax backward optimize

* rename sendrecv.proto to namespace paddle.distributed

* split ps with distributed

* fix elugradgrad test fail and error message opt

* fix unitest,test=develop

* Update prroi_pool_op.h

fix error message

* opt message,test=develop

* fix ci fail,test=develop

* Fix the accuracy problem of allclose op when using float64 data type in static mode.

* Format the code style.

* fix cmake
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

Support vector<double> as type of op attribute and op set_value suppport vector<double> as value (#30126)

* type promotion for grad

* add type promotion for div op

* modify error message based on comments

* edit code according to review.

* Correct spelling according to review.

 add the op error message for the matmul xpu 

* optimize softmax forward

reduce the  occupied size  of memory for the fused pattern of elementwise_add Op and activation Op(relu Op for example) (#29885)

* - Added UT for testing elementwise_mul caching

* lint fixes

* add cast ops before and after unsupported fp16 ops.

* Keep partial net in FP32 pattern.

* Support check_finite_and_unscale and update_loss_scaling for FP16 calculation mode.

* Add fp16 support for adam op.

* add multi precision attr for adam.

* Fix the bug of test_multi_precision_fp16_train UT.

* Code format for CI.

* Fix the redefine error about MPTypeTrait on windows.

* fix bugs of the _create_accumulators func in Momentum.

* fix bug when inserting post cast op.

* Add the update_loss_scaling op in allow_set of UnusedVarCheck.

* Update for ci coverage.

* Add some doc for OptimizerWithMixedPrecision.

* Fix the code style.

* Imporve the doc of `amp_init`.

* Change for fp16 testing if users have the infer program defined in separate way.

* fix dtype of ungenerated grad var

* update ut

* refine code

* set default dtype

* fix could_use_cudnn bug

* remove debug code

* re-implement

* fix bug

* add tensor table

* enhance error message of nll_loss op test=develop

Improve some error messages in parallel_executor.cc, conditional_block_op.cc, recurrent_op.cc

* test=develop, add index_select_cuda kernel

* enhance error info for py_func

* update