Add distribution supported

* test=develop, fix fleet.metrics(mse, rmse, mae)

* test=develop, add distributed_infer

As the title

As the title

* optimize memcpy perf for kunlun

* remove useless unitest for kunlun mean

* minor

* upgrade oneDNN version to 2.0 master branch

* - Added workarounds for new lib onednn change

* fix regex
Co-authored-by: NJacek Czaja <jacek.czaja@intel.com>

* disable test_pipeline

* fix error

* fix bug of using ignore_index and reduction,test=develop

* fix bug of celoss when using ignore_index and reduction, test=develop

* improve performance when ignore_index=-100, test=develop

* add test in test_cross_entropy_loss.py for coverage rate, test=develop

* rm comment in test_cross_entropy_loss.py, test=develop

* del  hard code of "float64" in python/paddle/nn/functional/loss.py, test=develop

* change mask to a more simplified implementation, test=develop

* del comment in python/paddle/nn/functional/loss.py, test=develop

* del hard code and change mask to a more simplified implementation, test=develop

* change mask to a more simplified implementation, test=develop

* change mask to a more simplified implementation, test=develop

* fix elugradgrad test fail and error message opt

* fix unitest,test=develop

* Update prroi_pool_op.h

fix error message

* opt message,test=develop

* fix ci fail,test=develop

* Fix the accuracy problem of allclose op when using float64 data type in static mode.

* Format the code style.

Support vector<double> as type of op attribute and op set_value suppport vector<double> as value (#30126)

* type promotion for grad

* add type promotion for div op

* disable ut test_tsm on windows

* fix error

* add ut execuate time

* Fix bug for 'save mutiple method'

* To pass coverage.

* edit code to pass coverage.

* edit code to pass coverage.

* add unittest for coverage.

* change for coverage.

* edit for coverage.

Add clone method for static Variable so that this interface will be same as dygraph. It fixed some bugs in dy2stat

* delete paddle.nn.functional.assign

* fix dynamic to static error

reduce the  occupied size  of memory for the fused pattern of elementwise_add Op and activation Op(relu Op for example) (#29885)

* add view strategy on squeeze,unsqueeze,reshape,flatten

* add squeeze unittest

* add unittests

* use View strategy as name rather than Reuse Allacation

* fix view api doc

* fix format

* use core.ops when input of reshape2 is Tensor

* fix test_cross_entropy_loss error because of reshape2

* delete selected_rows

* change op_function

* little change

* solve HandleViewBetweenInputAndOutput

* fix dtype of ungenerated grad var

* update ut

* refine code

* set default dtype

* fix could_use_cudnn bug

* remove debug code

* re-implement

* fix bug

* fix tensor shape bug

* fix op_num

* clean code

1. When x is Variable, call nn.shape(x) only in following cases:
 1）The shape of x is used in control flow condition.
 2）The dim to be used is negetive
2. When x is Variable, but x.shape or x.shape[idx] doesn't contain negetive value, don't convert to paddle.shape()

* change to tensor copy sync

* change to tensor copy sync

* make copy_to safe when use TensorCopy

* refine code

* add ut

* add cudapinned garbagecollector

* add testcase: cpu place -> cuda pinned place

In control flow, don't copy TensorArray from subblock to parent block when TensorArray is created in parent block.

* add tensor table

* simplify prepared op impl to improve performance

* fix kunlun compile error

* continue fix kunlun compile error

* only transform diff place when dtype diff

* fix failed unittests

* remove useless file

* polish impl by review comment