* add some basic functions

* add elementwise rule for auto parallel

* add unit test for elementwise rule

* fix the lib name in spmd rule test cmake file

* fix some bugs

* add unit tests for elementwise spmd rule in python

* bug fix

* delete cpp unit test for elementwise spmd rule (use python ut now)

* add cpp unit test for elementwise rule

* use concrete op name in unit test

* fix typo

* fix code style

* delete cpp unit test

* add more details in log

* add layernorm spmd rule

* add ut

* follow comments

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>

Early stop if return value of bkcl all is not BKCL_SUCCESS

* update dygraph collective

fix ut

* remove debug log

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

* test wrap DistTensorSpec in dygraph mode

* matmul main logic done

* define unified data class for inferencing dist_attr

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>

* new_ir to shared

* refine code

* add ir lib path to env

* refine type

* refine code

* fix bug

* fix bug

* refine code

* refine code

* close win

* refine code

* refine code

* refine code

* add win share

* refine code

* refie code

* refine code

* refine code

* refien code

* fix bug

* fix bug

* fix bug

* solve conflict

* solve conflict

* fix bug

* refine code

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* refine code

* fix interpretercore program bug

* delete unuse code

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix cinn bug

* fix cinn bug

* debug

* fix cinn bug

* delete unused code

* fix cinn bug

* fix cinn bug

* fix  ug

* test win openblas

* test win openblas

* fix win openblas bug

* polish code

* fix win open blas bug

* close win dll

* fix flag bug

* test for windows

* fix compile bug

* “update”

* update

* update

* update

* update

* test

* update

* test

* fix_c++17_bug

* fix coverage compile error

* test

* test

* test

* fix C++17 error

* fix c++17 error

* fix c++17 error

* test

* test

* test

* test

* fix cinn compile error

* compile to compiler

* set cinn c++14

---------
Co-authored-by: huangjiyi <947613776@qq.com>
Co-authored-by: Nhuangjiyi <43315610+huangjiyi@users.noreply.github.com>

This reverts commit 8b1839be.

* “update”

* update

* update

* update

* update

* test

* update

* [CustomDevice] add sarding support

* update

* [AutoParallel] update while control_flow with pipeline

* update process group instantiate

* fix micro_bsz for reshard

* update api for micro batch size

* add strategy for dp optimization

* update_c++17

* update_c++17

* fix windows bug

* solve cirle depend

* solve cirle depend

* solve cirle depend

* solve cirle depend

* solve cirle depend

* fix windows bug

* fix compiler error

* fix compiler error

* update eigen3

* update eigen3

* update eigen3

* fix mac-py3 compiler error

* update C++17

* fix mac compiler error

* fix compile error

* fix coverage_compiler error

* fix coverage_ci_problem

* fix coverage_error

* fix_kunlun200 compile error

* fix kunlun200 compiler error

* fix compile error

* fix compiler error

* fix py3 failed test

* fix kunlun200 compiler error

* test

* fix test error

* fix test error

* fix test error

* test

* test

* fix mac py3 error

* fix mac py3 error

* fix mac py3 error

* fix test error

* fix test error

* fix compile error

* fix compile error

* fix compile error

* test

* test

* fix compiler error

* test

* test

* debug on ci

* fix compiler error

* fix compiler error

* test

* fix cinn compiler error

* test

* fix rocm cmpile error

* fix cinn and kunlun compile error

* update c++14

* Update flags.cmake

* create phi so

* fix ci bugs

* fix py3 bugs

* add file

* fix py3 bugs

* fix windows bugs

* perfect so

* fix py3 bugs

* delete all static target in phi

* fix windows bugs

* fix py3 bugs

* fix ci bugs

* fix windows bugs

* fix bugs: gflags can't be linked by dynamic and static lib

* fix bugs that can not load 3rd party

* fix ci bugs

* fix compile bugs

* fix py3 bugs

* fix conflict

* fix xpu bugs

* fix mac compile bugs

* fix psgpu bugs

* fix inference failed

* deal with conflict

* fix LIBRARY_PATH bug

* fix windows bugs

* fix onednn error

* fix windows compile bugs

* fix windows compile bugs

* fix test_cuda_graph_static_mode_error aborted

* fix windows bugs

* fix mac-python3 error

* fix hip compile bugs

* change mode to static

* change to static mode

* fix ci bugs

* fix py3 bugs

* fix windows bugs

* fix bugs

* add static flag

* add PADDLE_API

* change position of PADDLE_API

* fix windows bugs

* change mode to dynamic lib

* fix windows static bugs

* deal with conflict

* fix windows unit bug

* fix coverage

* deal with conflict

* fix windows-inference

* fix py3 bugs

* fix bugs when compile type_info

* fix compile bugs

* fix py3 bugs

* fix windows bugs

* fix windows openblas

* fix xpu bugs

* fix enforce_test in windows

* update code according comment

* fix windows cmake bug

* fix windows bugs

* fix windows bugs

* delete cinn unittest

* fix cinn bugs

---------
Co-authored-by: lzydev <1528794076@qq.com>

* merge code from forsish

* polish

* paddle/fluid/pybind/auto_parallel_py.cc

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* [XPU] update log for bkcl function calls.

* minor update

* revert unnecessary modifications.

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* add flags for phi

* fix compile bugs

* fix ci bugs

* fix inference bugs

* fix cinn' bugs

* fix cinn bugs

* perfect code according comment

* fix ci bugs

* fix ci bugs

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test,test=develop

* test ,test=develop

* [CustomDevice] add pipeline parallel support

* update

* update

* optimize gap between generation step

* remove useless header

* [CustomDevice] add c_identity op

* fix use calc stream

* cherry-pick fleet executor from 2.4

* fix test case