* support CUDA Graph on PE

* add ut, fix CI compile

* reduce memory consumption

* fix CUDA 10 CI

* improve coverage

* improve python coverage

* fix feed, test=develop

* delete one test case, test=develop

* Enhance Check mechanism of fetch_list in Executor

* support single tuple

* fix typo

* fix typo

Intergrate StandaloneExecutor in Static.Executor Interface with FLAGS_USE_STANDALONE_EXECUTOR (#35628)

* Intergrate StandaloneExecutor in Static.Executor Interface with FLAGS_USE_STANDALONE_EXECUTOR

* Enhance unittest and clean code in StandaloneExecutor

* polish unittest

This PR added optional boolean is_parameter and stop_gradient in the VarDesc proto, and remove them during save_inference_model

* add trainprofiler for heterps in oneps; test=develop

* add set_use_ps_gpu; test=develop

solve hccl communicate conflict (#32447)

* pscore support heterps

* fleet cmake

* fleet wrapper

* macro

* solve conflict

* solve conflict

* add unitest

* paddle enforce

* unitest

* unitest

* unitest

* [feature] support npu allocator (#30840)

[feature] support npu allocator

* [feature] support npu operator (#30951)

[feature] support npu operator

* [feature] support npu allocator, part 2 (#30972)

* support npu allocator

* add npu device context

* fix some compile problem

* fix some compile problem

* add npu info

* compile ok

* fix include dir

* support naive_best_fit_allocator

* run ut ok, bug failed to exit

* call aclrtResetDevice before exit

* fix aclFinilize

* add system allocatot test

* add selected_gpus in gtest

* add tensor_test for npu

* support npu op, initial commit

* add npu stream

* add elementwise_add_op

* compile ok

* fix typo

* fix elementwise_add_op_npu_test

* support op run

* test can run but failed

* change aclopExecuteV2 to aclopCompileAndExecute

* support parsing ascend rank table file (#31000)

support parsing ascend rank table file

* Fix reshape on GE graph. (#31084)

Fix reshape on GE graph

* add npu kernel for elementwise_sub and elementwise_sub_grad (#30973)

* add npu sub op

* fix typo

* rename test

* fix bug

* fix bug

* add fp16 kernel

* fix typo

* support sub grad op

* support elementwise_sub_grad op
Co-authored-by: Nfrankwhzhang <frankwhzhang@126.com>

* Fix compilation problem (#31100)

Fix compilation problem (#31100)

* fix compile

* fix code stype

* remove const_cast

* support adding correct npu op in pybind.h (#31143)

* support adding correct npu op in pybind.h

* refine code

* [NPU] Support executor with NPU (#31057)

* [NPU] Support executor with NPU

* Fix code according to reviews

* Fix code

* Add unittest for sub op npu

* refactor npu device manager (#31154)

refactor npu device manager (#31154)

* fix selected npus

* fix compile

* fix reading flags from env

* format
Co-authored-by: Nxiayanming <41795079@qq.com>
Co-authored-by: Ngongweibao <weibao.gong@gmail.com>
Co-authored-by: Nfrankwhzhang <frankwhzhang@126.com>
Co-authored-by: Nliym27 <33742067+liym27@users.noreply.github.com>

* update, test=develop

* 　add heter box

* add trainer, worker, wrapper...

* format

* for ci

* format

* remove boost get

* boost & copyright

* rename

* 　rename

* format

* format

* format
Co-authored-by: Nyaoxuefeng6 <yaoxuefeng@baidu.com>

* update, test=develop

* ps gpu transpile

* ps gpu

* remove op

* gps trainer

* local ps

* add macro

* HeterBox

* def cuda

* tab

* code style

* style

Co-authored-by: Thunderbrook <a754913769#163.com>

* update version pybind to v2.4.3, test=develop

* update unittests, test=develop

As the title

* fix doc and unittest of 2.0 lr_scheduler

* fix doc of 2.0 lr_scheduler

* fix unittest

* fix english doc of lr_scheduler

* fix api name of lr scheduler

* fix api name of lr scheduler

* add xpu in heter mode
test=develop

* BOOST_CONST_GET; PADDLE_THROW
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* refine
test=develop

* refine
test=develop

* refine
test=develop

* refine code
test=develop

* optimized transformation form tensor to numpy, test=develop

* optimized transformation form tensor to numpy, pass pre-commit, test=develop

* modify fetchophandle zerocopy to deepcopy in PE&CUP, test=develop

* modify py:array construct, test=develop

* fix _fetch_var to use deep copy, test=develop

* Reconstruct all API related to lr scheduler, unify dygraph and static

* Reconstruct all API related to lr scheduler, unify dygraph and static

* fix doc

* fix doc

* fix doc of lr_scheduler

* fix unittest and english doc

* fix english doc

* fix confilt

* fix doc

* add check if fluid.data() variable no feed data, test=develop

* Add testcase for feed check, test=develop

* add heter ps mode

* code style
test=develop

* add with_pslib
test=develop

* unitest
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* test monitor
test=develop

* prepare trainer
test=develop

* code style
test=develop

As the title

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features

* hidden the explicit setting of dataset for pipeline training.

* refine as_lodtensor, test=develop

* fix test, test=develop

* add unittest, test=develop

* handle nested_list, test=develop

* handle nested_list, test=develop

* test=develop, test=document_fix

* test=develop, test=document_fix
Co-authored-by: Nswtkiwi <1208425345@qq.com>

* Support LoDTEnsorArray in fetch op

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

* Support LoDTensorArray in fetch

test=develop

test=develop

* fix init_gflags with 'python -c', test=develop

* fix pruned_program_cache_key of Operator, test=develop