* add embedding backward rule

* update backward api

* revert api

* matmul inferbackward

* update unitest

* fix small bug in reshape spmd rule

* small fix in unit test

* add reshape spmd rule

* add unit test for reshape spmd rule

* bug fix

* replace the print_info function with to_string

* fix typo

* bug fix

* add handling for "0" in target shape

* remove the part of computing size in dim_trans.cc

* [clang-tidy] enable modernize-use-emplace

* Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into modernize_use_emplace

* [Semi-Auto] Add transpose spmd rule

* add unit test in cmake file

* log perm info

* add split spmd rule

* add pytest in cmake file

* small fix

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

* test wrap DistTensorSpec in dygraph mode

* matmul main logic done

* shape int64

* common cc

* define unified data class for inferencing dist_attr

* test wrap DistTensorSpec in dygraph mode

* define python api and wrap function in static mode for DistTensorSpec

* revise syntax

* map bugfix

* broadcast func

* compile 1

* add unitest

* add registry

* update unitest

* bugfix

* bugfix

* add pybind

* bugfix

* bugfix macro gloabl name space

* bugfix macro gloabl name space

* pybind

* pybind test

* pybind bugfixed1

* pybind bugfixed2

* pybind unitest

* merge dev

* merge dev

* merge dev

* fixed cmake conflict

* fixed cmake conflict

* rename get method

* revise inferforward output type

* revise comment

* replicated rule

* replicated rule 2

* revert bug deps

* add rule

* add unitest

* add rule

* add unitest

* move ut of auto_parallel

* fix ut

* bugfix

* bugfix

* bugfix

* bugfix

* bugfix

* bugfix

* bugfix

* resolute input sharding conflict maybe

* fixed comment

* add rule

* add unitest

* fixed typoes

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>
Co-authored-by: Nzhiqiu <chenqiuliang@baidu.com>

* resolute input sharding conflict maybe

* fixed comment

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>
Co-authored-by: Nzhiqiu <chenqiuliang@baidu.com>

* move ut of auto_parallel

* fix ut

* add reduction spmd rule for auto parallel

* fix the logic of handling partial

* fix code style

* fix the partial handling

* add some basic functions

* add elementwise rule for auto parallel

* add unit test for elementwise rule

* fix the lib name in spmd rule test cmake file

* fix some bugs

* add unit tests for elementwise spmd rule in python

* bug fix

* delete cpp unit test for elementwise spmd rule (use python ut now)

* add cpp unit test for elementwise rule

* use concrete op name in unit test

* fix typo

* fix code style

* delete cpp unit test

* add more details in log

* add layernorm spmd rule

* add ut

* follow comments

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

* test wrap DistTensorSpec in dygraph mode

* matmul main logic done

* define unified data class for inferencing dist_attr

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>

* create phi so

* fix ci bugs

* fix py3 bugs

* add file

* fix py3 bugs

* fix windows bugs

* perfect so

* fix py3 bugs

* delete all static target in phi

* fix windows bugs

* fix py3 bugs

* fix ci bugs

* fix windows bugs

* fix bugs: gflags can't be linked by dynamic and static lib

* fix bugs that can not load 3rd party

* fix ci bugs

* fix compile bugs

* fix py3 bugs

* fix conflict

* fix xpu bugs

* fix mac compile bugs

* fix psgpu bugs

* fix inference failed

* deal with conflict

* fix LIBRARY_PATH bug

* fix windows bugs

* fix onednn error

* fix windows compile bugs

* fix windows compile bugs

* fix test_cuda_graph_static_mode_error aborted

* fix windows bugs

* fix mac-python3 error

* fix hip compile bugs

* change mode to static

* change to static mode

* fix ci bugs

* fix py3 bugs

* fix windows bugs

* fix bugs

* add static flag

* add PADDLE_API

* change position of PADDLE_API

* fix windows bugs

* change mode to dynamic lib

* fix windows static bugs

* deal with conflict

* fix windows unit bug

* fix coverage

* deal with conflict

* fix windows-inference

* fix py3 bugs

* fix bugs when compile type_info

* fix compile bugs

* fix py3 bugs

* fix windows bugs

* fix windows openblas

* fix xpu bugs

* fix enforce_test in windows

* update code according comment

* fix windows cmake bug

* fix windows bugs

* fix windows bugs

* delete cinn unittest

* fix cinn bugs

---------
Co-authored-by: lzydev <1528794076@qq.com>

* merge code from forsish

* polish

* paddle/fluid/pybind/auto_parallel_py.cc

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* Support stream priority for standalone executor

* Fix compile error

* Fix compile error

* Fix compile error

* Fix compile error

* Fix compile error

* Enable standalone executor for fleet training

* Update code

* Replace use_standalone_executor utils in auto parallel

* Update code

* Diable standalone executor for test_pass_sharding

* Update code

* Set sequential run for auto parallel

* Fix dist_attr bug

* Set sequential run for auto parallel

* [Auto Parallel] Rename methods of ProcessMesh

* [Auto Parallel] Impl the python process_mesh by the c++ one

* [Auto Parallel] Add some minor modifications

* [Auto Parallel] Rename some methods

* [Auto Parallel] Remove unnecessary codes

* [Auto Parallel] Add back some removed files

* [Auto Parallel] Fix bugs

* [Auto Parallel] Fix a bug

* Update process_mesh.cc

* [Auto Parallel] Merge dist attrs of Python into C++

* [Auto Parallel] Add back deleted importing

* [Auto Parallel] Add back removed unittest

* [Auto Parallel] Remove type qualifiers of return types

* [Auto Parallel] Fix some bugs

* [Auto Parallel] Fix a bug of the quant pass

* [Auto Parallel] Fix the code style

* Support priority scheduling for standalone executor

* Add CPU test

* [Auto Parallel] Improve the c++ dist attr

* [Auto Parallel] Modify test_program.py

* Support custom stream for standalone executor
Co-authored-by: NYulong Ao <aoyulong@baidu.com>

* [Auto Parallel] Improve the c++ dist attr

* [Auto Parallel] Modify test_program.py

* [Auto Parallel] Add the missiong import

* [Auto Parallel] Pybind11 ProcessMesh and DeviceMesh

* [Auto Parallel] Fix the unittest problem

* [Auto Parallel] Explicitly add the src file for auto_parallel target

* [Auto Parallel] Add the proto depedency explicitly

* [Auto Parallel] Fix the cmake bug on windows and mac

* [Auto Parallel] Remove the pybind11 header file in process_mesh.h

* [Auto Parallel] Add the c++ dist attrs

* [Auto Parallel] Remove some codes to be less than 1000 lines

* [Auto Parallel] Add ProcessMesh, DeviceMesh and DistributedMapper

* [Auto Parallel] Remove unecessary codes

* [Auto Parallel] Comment out unnecessary cmake statements