* Support both use_calc_stream and sync_op in send recv APIs (#46023)

* add batch_norm prim2orig rule
Co-authored-by: NWen Sun <35923278+HermitSun@users.noreply.github.com>

[Eager, Performance optimization] math op sink to Cpp level ( + and - operator as an example ) (#45811)

* [Eager] math op sink to Cpp level

* fix ci errors

* draft version

* draft version

* draft version

* support + and - operator under cpp directly

* add static test

* polish code

* promote types or unify right type to left

* recover static test case

* polish code and fix some ci errors

* support complex and polish code

* fix conflicts

* fix windows ci errors

* fix windows-inference-ci errors

* polish and fix tests

* fix test case

* polish code

* polish code

* polish code and fix code-format

* polish code

* polish code

* polish code

* polish code

Signed-off-by: NZhang Jun <jzhang533@gmail.com>
Signed-off-by: NZhang Jun <jzhang533@gmail.com>

* enable eager mode on xpu, test=kunlun

* add numpy support to xpu

* fix tensor using error

* fix  error, test=kunlun

* fix failed tests, test=kunlun

* support rsqrt_p

* refine code and ut

* add_prim_rsqrt

* fix ut

* open the clip_extra flag in paddle.static.save_inference_model, test=allcase

* set the defalut value of clip_extra in TracedLayer from False to True, test=allcase

*  Disable TF32 to solve accuracy for test_trt_conv_pass and test_trt_deformable_conv in A10 GPU.

* Add ExecutionConfig and fix last-live-op bug for standalone executor

* Improve code design

* remove `yapf: disable` from math.py

* disable test case only

* remove useless disable

* remove useless disable

* revert no_grad_set_white_list

* Optimize performance of depthwise_conv_fwd

* fix

* add phi reduce_sum test=kunlun

* add fhi reduce_sum test=kunlun

* add fhi reduce_sum test=kunlun

* 增加ks-test；注册kl_divergence

* bug fix

* bug fix data shape

* 按要求修改

* sample code fix

* 按要求修改

* bugfix

* bugfix

* bugfix

* bugfix

* bugfix

* fix sample func

* fix sample func

* 按要求修改；sample函数bugfix

* 按要求修改；sample函数bugfix

* bugfix

* 补充英文文档公式

* 英文文档bugfix

* bugfix

* bugfix

* 英文文档修复

* 去除name参数

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fl-ps v1.0

* .

* support N + N mode

* .

* .

* .

* .

* delete print

* .

* .

* .

* .

* fix bug

* .

* .

* fl-ps with coordinator ready

* merge dev

* update message parse only

* update fl client scheduler

* fix bug

* update multithreads sync

* fix ci errors

* update role_maker.py

* update role_maker.py

* fix ci error: windows py import error

* fix ci error: windows py import error

* fix windows ci pylib import error

* add dump fields & params

* try to fix windows import fleet error

* fix ps FLAGS error

* fix logging risk

* fix logging possible risk

* write trainer_desc file

* support split sparse params in local & remote

* fix import paddle.fluid.core.PSGPU

* fix import paddle.fluid.core.PSGPU

* add remote_sparse & local_sparse config

* fix unittest

* fix test_dist_fleet_geo table error

* fix PADDLE_ENFORCE error

* fix other's pr conflict

* forbidden ssd table

* .

* recover ssd table code

* recover file mode

* debug auc 0.5

* adapt for nn fl-ps

* adapt for nn fl-ps

* add learning_rate_0 intializer op

* recover ssd table

* modify file mode

* flps del fake-init op

* bug fix

* first commit

* clarify the quotes

* change code style format

* support bfloat16

* [BugFix]Fix reduce_mean/min/sum/prod, cumsum grad_op infershape bug

* fix typo

* fix typo

* 1. Delete some expression like 'This Op'
2. remove import numpy as np

* test=document_fix

* fix eg; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix some docs; test=document_fix

* delete raise; test=document_fix

* add some introduction; test=document_fix

* add some introduction; test=document_fix

* test=document_fix

* Fix ’note‘ format; test=document_fix

* Fix Returns of cholesky; test=document_fix

* Fix Example format; test=document_fix

* Fix det; test=document_fix

* Fix eig; test=document_fix

* Fix eigh; test=document_fix

* Fix eigh; test=document_fix

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* test=document_fix

* test=document_fix

* KLDiv;test=document_fix

* norm example code; test=document_fix

* revert python/paddle/fluid/**/*

* revert python/paddle/distributed/spawn.py

* revert python/paddle/fluid/*

* fix a `Note` format

* Fix inv; test=document_fix

* Fix lu; test=document_fix

* Fix lu_unpack; test=document_fix

* Fix matrix_power; test=document_fix

* Fix multi_dot; test=document_fix

* Fix solve; test=document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>