* [Auto Parallel] Pybind11 ProcessMesh and DeviceMesh

* [Auto Parallel] Fix the unittest problem

* [Auto Parallel] Explicitly add the src file for auto_parallel target

* [Auto Parallel] Add the proto depedency explicitly

* [Auto Parallel] Fix the cmake bug on windows and mac

* [Auto Parallel] Remove the pybind11 header file in process_mesh.h

* add init file

* add op definition and infermeta

* add kernel definition funcs

* add broadcast infer shape

* add gpu forward kernel

* delete SUB and DIV

* add x_grad

* add template

* add e_grad for min and max

* fix small bug

* temp commit

* temp commit

* add e_grad for sum and mean

* fix some compile bug

* fix compile bugs

* fix compile problem

* add sum forward unittest

* fix broadcast error, add kernel sig, register e_grad, change unit test

* fix grad

* add temp grad fix

* temp commit

* add min max unittest

* add max, min unittest, fix mul bug

* add cpu forward sum and mean

* add forward min max, fix mean unittest

* add cpu backward min max

* fix code-style

* add backward sum mean

* fix rocm ci

* set uniitest timeout

* fix bug of x broadcast to e, gpu grad

* fix bug of x broadcast to e, cpu grad

* rename BOOST_GET_CONST macro

* fix rocm ci

* mv graph_send_e_recv to graph_send_ue_recv

* move out_size to IntArray

* add eager op test

* fix max pool type bug, add unittest for api

* revise api doc

* add fp16 for atomic min and max, add unittest

* add unittest

* add fp16 support for graph_send_recv

* fix unittest fp16 bug

* change OutSizeTensor to Out_size

* move E to Y

* add copyright, fix comment

* review code

* fix thread block size

* fix thread block size

* change api attribute name: pool_type to reduce_op, compute_type to message_op

* change api attribute name, move pool_type to reduce_op, move compute_type to message_op

* first refine

* second refine

* remove some code unuseful

* fix

* add control flag and input shapes for new dygraph

* fix file mode

* improve code coverage

* fix a bug in statstic

* fix according to review

* optimize performance

* fix

* fix proto bug

* add ut

* reset need_update for var_desc

* refine code

* fix var desc order issue

* [OpAttr]Support VarDesc* and vector<VarDesc*> in Attribute

* add unittest for inference predictor

* [Eager] use final_state_fill_constant_

* fill_constant use str_value

* add fill_constant_ to no_amp_list

* use float(value) as input

* support final state full_ same as fill_constant

* change out_size to INTArray

* fix out_size eager bug

* add unittest for out_size tensor

* add deprecated for paddle.incubate.graph_send_recv, add paddle.geometric.send_u_recv and unittests

* fix lowest bug

* fix according review comment

* add default value in yaml

* change api file name

* change name

* Pybind Fucniton and hide ExecutorEngine and PEEngine

* Remove FunctionNames in compilation_unit

* Polish function code

* Rename funciton to engine

* Fix Log msg and doc

* Rename Function to Engine and using new Function class to warp Engine

* Rename EngineInfo

* Adjust member variable order

* [Eager] fix to_uva_tensor without specific gpu number

* Update test_tensor_uva.py

update test case

* update trt workspace size param

* update

* update

* update

* use int64_t

* use int64_t

* upate

* update

* Move Function classes to a sub dir

* Format code

* [CustomDevice] add custom ccl 2/2

* update

* update

* update launch

* [Eager] call final_state_slice under eager mode

* rm useless comments

* use eager final state instead intermidiate state

* update fill_constant yaml

* update fill_constant yaml

* modify wrapped_infermeta_gen logic to fix special case

* fix slice in manipulation

* use fill_constant_

* modify slice infermeta

* rm final_state_conv2d

* use final_state_slice

* use final_state_slice only

* polish slice, use final state

* add paddle_throw for SplitInferMeta

* rm fill_constant_ temply

* recover array_equal, not allclose

* recover original code

* remove cudaDeviceContext

* remove more template

* fix rocm compile

* remove alias name CUDADeviceContext

* fix compile

* fix tests

* revert changes

Co-authored-by: seemingwang <zsasuke@qq.com>
Co-authored-by: NDesmonDay <908660116@qq.com>
Co-authored-by: Nseemingwang <seemingwang@users.noreply.github.com>
Co-authored-by: NThunderbrook <a754913769@163.com>
Co-authored-by: Nxuewujiao <105861147+xuewujiao@users.noreply.github.com>
Co-authored-by: Nroot <root@yq01-sys-hic-k8s-v100-box-a225-0693.yq01.baidu.com>
Co-authored-by: NThunderbrook <52529258+Thunderbrook@users.noreply.github.com>
Co-authored-by: Nroot <root@yq01-inf-hic-k8s-a100-ab2-0009.yq01.baidu.com>
Co-authored-by: Nhuwei02 <53012141+huwei02@users.noreply.github.com>
Co-authored-by: Nyaoxuefeng <yaoxuefeng@baidu.com>
Co-authored-by: Nlxsbupt <luoxsbupt@163.com>
Co-authored-by: Nmiaoli06 <106585574+miaoli06@users.noreply.github.com>
Co-authored-by: Nroot <root@yq01-inf-hic-k8s-a100-ab2-0008.yq01.baidu.com>
Co-authored-by: Nchao9527 <33347532+chao9527@users.noreply.github.com>
Co-authored-by: Nqingshui <qshuihu@gmail.com>
Co-authored-by: Nyangjunchao <yangjunchao@baidu.com>

* migrate lstsq op

* update

* fix bugs for CIs

* update

* fix bugs

* add uts

* update

* update

* update

* fix bugs of jip

* fix bugs of hip

* update

* update according to review

* update

* update

* update

* update

* init

* move CUDAStream to phi

* fix compilation

* merge develop

* add stream_owned_ member

* split cuda_stream.h

* fix cpu compile

* fix constructor

* fix bug

* fix windows compile

* fix inference test_levit

* fix windows tests

* [Phi] Add yaml for assign_value

* [Phi] Fix the bug of the assign api and modify the unittest

* [Phi] Fix the bug when the tensor does not have the backend info

* [Phi] Replace the functional-style cast init by the brace-init

* [Phi] Cast the data explicitly

* fix RemoveNode in fuse_elewise_add_act_pass

* fix

* change pointer to share_ptr

* fix

* fix

* fix format

* fix

* fix graph_safe_remove_nodes

* jit.save support peropty serilization

* extract set property function

* fix property test file name

* fix typing error

* fix typing error

* fix test coverage

* sync misc changes

* add authors
Co-authored-by: NZhaorui Chen <zhaoruic@graphcore.ai>

* up x

* Revert "up x"

This reverts commit f3fde458c6cc48613269a643cfe2acf689caccd3.

* add guarg for ipu
Co-authored-by: NZhaorui Chen <zhaoruic@graphcore.ai>

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fl-ps v1.0

* .

* support N + N mode

* .

* .

* .

* .

* delete print

* .

* .

* .

* .

* fix bug

* .

* .

* fl-ps with coordinator ready

* merge dev

* update message parse only

* update fl client scheduler

* fix bug

* update multithreads sync

* fix ci errors

* update role_maker.py

* update role_maker.py

* fix ci error: windows py import error

* fix ci error: windows py import error

* fix windows ci pylib import error

* add dump fields & params

* try to fix windows import fleet error

* fix ps FLAGS error

* add property.proto, can compiled

* property get and deserilize

* support get float

* format code

* format code

* add unittest

* add more set method

* fix grammar error

* Update paddle/fluid/jit/property.h
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* Update paddle/fluid/jit/property.cc
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* Update paddle/fluid/jit/property.cc
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* Update paddle/fluid/jit/property.cc
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* fix comment

* fix error throw

* fix property save unit test

* fix error info

* fix copyright and header import

* reorder jit property tensor datatype
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* Support predictor function in JitLayer

* Pybind PEFunction

* Pybind PEFunction and call phi api in layer_test

* Call sqrt phi API

* Polish flags

* Fix comments

* fix FleetWrapper initialize

* release gil when run backward

* add save_cache/patch

* add pybind

* remove pybind

* remove const_cast

* add fleet

* Rename BOOST_GET macros

* Fix conflicts