* move momentum, rmsprop to phi; test=develop

* update

* update

* update

* update

* udpate; test=develop

* fix xpu npu bugs; test=develop

* fix npu bug; test=develop

* fix windows compile error; test=develop

* fix windows compile error; test=develop

* polish code; test=develop

* fix conflict; test=develop

* add meshgrid;

* update

* polish code

* polish code;

* fix bug

* format; remove useless code

* fix npu bug

* fix bug

* update

* add forward case

* update

* update; test=develop

* add some grad kernel; test=develop

* move gpu kernel; test=develop

* update

* update;

* update test;

* fix selected rows bug;

* add mix vector include ;

* add mixed vector depen; test=develop

* add logit grad signature;

* polish code

* fix bug;

* add namespace for abs

* revert code

* not move softsign

* revmove duplate register;

* fix softsign bug

* polish code

* format

* format

* fix bug

* remove cmake dep

* add square sqrt selected rows support

* update

* remove clip norm

* add standalone executor sqrt dep

* standalone exec denp sqrt

* remove sqrt op in cmkaelist

* open some case

* add uts for EagerReducer

* add more uts

* fix bugs

* fix bugs

* modify

* modify uts

* fix bugs

* update

* update

* update

* solve conflicts and merge

* add some other uts

* modify time of uts

* update

* update

* update

* remove uts of resnet

*  add FP16 Pass 

* Support the auto completion of while_op

*  acc aligned

* no

* fix bugs

* fix doc according to review

* fix api doc format

* fix api doc according to review

* fix bug and add unit test

* fix record event bug

* fix code stype

* update unitest

* [Eager] Support SelectedRows in eager mode

* Remove unnecessary codes

* Adapt new dygraph flag

Enabled eager_mode for complex unit tests, except for test_complex_op.py and test_complex_view_op.py (#40887)

* [Dy2Stat] Fix ForLoop Transformation with single return

* [Dy2Stat] Fix ForLoop Transformation with single return

* Refine test_lac.py for eager mode

* refine code

* Fix test_program_translator for eager

* Fix bug while specifying target grad in high order gradient

* add more unittest

* add more unittest

* first version, maybe many errors

* refactor op_test

* fix compare list

* fix bg

* fix bugs

* fit for mkldnn and inplace op

* fix compile

* refine ut

* register op version

* fix inplace op

* fix transfer_layout

* move slice to pten

* merge develop; test=develop

* fix slice bug;

* update

* update

* fix error

* update

* fix bug

* polish code

* polish code

* polish code

* try to fix windows bug

* add gpu compile flag;

* try to fix

* remov template;

* polish code;

* fix npu bug;

* fix npu bug

* fix npu bug; test=develop

* fix slice bug;

* remove no need dep

* [NPU] fix npu cast ut

* [NPU] fix npu cast ut

* update

* remove useless code

* remove label smooth test

* polish code

* polish code

* polish code

* remove _in_eager_mode error;

* add cast_grad phi kernel

* refie unittest

* refien unittest

* refine unittest

* refine include header path

* refien xpu cast unittest

* refine code

* support multi_dims for tril_triu, *test=kunlun

* support multi_dims for tril_triu, *test=kunlun

* support multi_dims for tril_triu, *test=kunlun

* update xpu.cmake date, support multi_dims for tril_triu, *test=kunlun

* refactor eager flags

* fix flags error when we switch from eager to dygraph

* fix ci problem

* fix ci

* fix ci

* merge develop and fix code style

* merge develop and fix code style

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* merge develop

* 1. add the python api grad 2. add final and intermediate state vlog 3. change the python_api error logic

* add python api or close the check_eager=True

* fix the compatibility

* matmul

* disable unittests: test_elementwise_add_op test_scatter_nd_op test_gather_nd_op test_scatter_op test_index_sample_op test_elementwise_add_mkldnn_op

* refine the logic of prepara_parameter logic

* fix Tensor(gpu) 2 Scalar segment fault.

* add multi-attribute. (test_unsqueeze_op); add python_sig_out for customizing op sig out

* fix some bugs, support python_out_sig

* [NPU] add merged_momentum

* fix

* fix device

* support get_item where the index is a bool scalar tensor

* add unittests for supporting get_item where the index is a bool scalar tensor

* approve amp for intermediate_dygraph

* add amp_utils for intermediate_dygraph

* add amp needcast check for mlu & npu

* test unittest

* add SetGradNode for set_stop_gradient && add checktensor for GradientHooks

* refine code

* refien unittest of imperative_amp for new dygraph

* inplace api skip amp

* add test_imperative_qat_amp for intermediate amp

* refine code

* refine test_amp ci strategy

* refine unittest code

* refine amp_utils code

* refine amp getpromotetype for some special op

* refine unittest code

* # This is a combination of 10 commits.
# The first commit's message is:
add expert count op

add ut for expert_count

# This is the 2nd commit message:

update UT only for cuda

# This is the 3rd commit message:

fix for rocm

# This is the 4th commit message:

update ut

# This is the 5th commit message:

add moe module

# This is the 6th commit message:

add expert count op

add ut for expert_count

# This is the 7th commit message:

update UT only for cuda

# This is the 8th commit message:

update ut

# This is the 9th commit message:

add moe module

# This is the 10th commit message:

make expert count private

* add assign pos op

* fix upper num name

* add api _assign pos

* add ut for assign pos op

* update date

* fix for win

* update for test (timeout)

* fix ut

* update

* fix ut for number count
Co-authored-by: Nhlygit66666 <2570058140@qq.com>

* [Auto Parallel] gradient merge pass support dist attribute

Polish optest: refine the optest parameter logic. support name, dtype, out, output in arbitrary position (#40824)

* 1. add the python api grad 2. add final and intermediate state vlog 3. change the python_api error logic

* add python api or close the check_eager=True

* fix the compatibility

* matmul

* disable unittests: test_elementwise_add_op test_scatter_nd_op test_gather_nd_op test_scatter_op test_index_sample_op test_elementwise_add_mkldnn_op

* refine the logic of prepara_parameter logic

* fix Tensor(gpu) 2 Scalar segment fault.

* Refine eager run_program OP for dy2st UT

* append run_program error string and refine run_program_grad

* remove some comments

* refine ConstructXGradTensors

* refactor cost model

* added missing BF16 activations

* added softplus bf16

* minor change

* disabled tests for GPU