* add random routing op

add _random_routing api in utils

add random routing ut

* # This is a combination of 10 commits.
# The first commit's message is:
add expert count op

add ut for expert_count

# This is the 2nd commit message:

update UT only for cuda

# This is the 3rd commit message:

fix for rocm

# This is the 4th commit message:

update ut

# This is the 5th commit message:

add moe module

# This is the 6th commit message:

add expert count op

add ut for expert_count

# This is the 7th commit message:

update UT only for cuda

# This is the 8th commit message:

update ut

# This is the 9th commit message:

add moe module

# This is the 10th commit message:

make expert count private

* add assign pos op

* fix upper num name

* add api _assign pos

* add ut for assign pos op

* update date

* add op about moe gate

update utils

add limit by capacity op

add ut for limit_by_capacity

add ut for prune_gate_by_capacity

add ut for limit_by_capacity

add ut for prune_gate_by_capacity

* fix for win

* fix bugs in test_limit_by_capacity_op

* update ut

* update for test (timeout)

* fix ut

* update

* update(fix) ut for win

* moe apis in incubate

* # This is a combination of 10 commits.
# The first commit's message is:
add expert count op

add ut for expert_count

# This is the 2nd commit message:

update UT only for cuda

# This is the 3rd commit message:

fix for rocm

# This is the 4th commit message:

update ut

# This is the 5th commit message:

add moe module

# This is the 6th commit message:

add expert count op

add ut for expert_count

# This is the 7th commit message:

update UT only for cuda

# This is the 8th commit message:

update ut

# This is the 9th commit message:

add moe module

# This is the 10th commit message:

make expert count private

* add assign pos op

* fix upper num name

* add api _assign pos

* add ut for assign pos op

* update date

* fix for win

* update for test (timeout)

* fix ut

* update

* fix ut for number count

* add apis and utils

* add gate apis

* add moe and grad clip apis

* update moe apis

* add ops for moe gate

* fix

* update for base moe layer api

* add random routing op

add _random_routing api in utils

add random routing ut

* fix for dygraph

* update with ranodm routing

* update

* fix ut for limit by capacity

* update
Co-authored-by: Nhlygit66666 <2570058140@qq.com>

* add elementwise sub and elementwise div in tensorrt op teller

* add unittest of elementwise mul, sub and div

This reverts commit 9c0eaada.

This reverts commit b532315d.

* Fix test_reinforcement_learning.py for eager run_program OP

* Add comment

This reverts commit e77a947e.

* [Phi] trans logsumexp op

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* add sig

* fix sig bugs

* fix sig bugs

* fix xpu bugs

* fix review bugs

* test=develop

* mv floordiv to phi

* mv elementwise_pow to phi

* fix as review

* Use _C_ops.yolov3_loss in eager mode for test_yolov3.py

* fix code for test_yolov3_loss_op

* remove useless import

* Fix dygraph_mode flag

[DoubleGrad PR #2] Adjusted logics of GenerateNodeCreationCodes and GenerateForwardDefinition (#41016)

* [Refactor] refactored eager_gen.py PR #2

* [DoubleGrad PR #1] Decoupled code generation logics for Dygraph ForwardFunctions and GradNodes

* Fixed minor issue

* Adjusted logics of GenerateNodeCreationCodes and GenerateForwardDefinition

* Fixed issues

* Fixed minor issue

* move momentum, rmsprop to phi; test=develop

* update

* update

* update

* update

* udpate; test=develop

* fix xpu npu bugs; test=develop

* fix npu bug; test=develop

* fix windows compile error; test=develop

* fix windows compile error; test=develop

* polish code; test=develop

* fix conflict; test=develop

* add meshgrid;

* update

* polish code

* polish code;

* fix bug

* format; remove useless code

* fix npu bug

* fix bug

* update

* add forward case

* update

* update; test=develop

* add some grad kernel; test=develop

* move gpu kernel; test=develop

* update

* update;

* update test;

* fix selected rows bug;

* add mix vector include ;

* add mixed vector depen; test=develop

* add logit grad signature;

* polish code

* fix bug;

* add namespace for abs

* revert code

* not move softsign

* revmove duplate register;

* fix softsign bug

* polish code

* format

* format

* fix bug

* remove cmake dep

* add square sqrt selected rows support

* update

* remove clip norm

* add standalone executor sqrt dep

* standalone exec denp sqrt

* remove sqrt op in cmkaelist

* open some case

* fix dependency

* reduce warnings

* trainer and worker

* delete commonsparsetable from gpups

* delete vlog

* codestyle

* delete communicator from gpups

*  add FP16 Pass 

* Support the auto completion of while_op

*  acc aligned

* no

* fix bugs

* fix doc according to review

* fix api doc format

* fix api doc according to review

* fix bug and add unit test

* fix record event bug

[DoubleGrad PR #1] Decoupled code generation logics for Dygraph ForwardFunctions and GradNodes (#40937)

* [Refactor] refactored eager_gen.py PR #2

* [DoubleGrad PR #1] Decoupled code generation logics for Dygraph ForwardFunctions and GradNodes

* Fixed minor issue

* remove in_dtype, out_dtype in redcue grad

* set the dtype and layout in noneedbufferInputs func

* So Parser

* add macro

* add macro

* slotrecord

* add macro

* code format

* add fused_seqpool_cvm op;test=develop

* moving OP

* move forward

* move grad and infershape

* code format

* format code

* fix code

* fix code

* fix CMakerLists.txt

* fix comments

* Refine CMakeLists for rocm ci

* [Eager] Support SelectedRows in eager mode

* Remove unnecessary codes

* Adapt new dygraph flag

* move assign value op kernel

* remove original op

* fix failed unittests

* remove infershape

* perfect reshape kernel

* fix bugs of sig

* add unittest for reshape_sig

* fix bugs when run converage

* Refine test_lac.py for eager mode

* refine code

* Fix test_program_translator for eager

* fit for mkldnn and inplace op

* fix compile

* refine ut

* register op version

* fix inplace op

* fix transfer_layout

* support check data type and mutable_data with compiled infos in paddle with cinn

* update cinn_instruction_run_op_test with multi data type

* move slice to pten

* merge develop; test=develop

* fix slice bug;

* update

* update

* fix error

* update

* fix bug

* polish code

* polish code

* polish code

* try to fix windows bug

* add gpu compile flag;

* try to fix

* remov template;

* polish code;

* fix npu bug;

* fix npu bug

* fix npu bug; test=develop

* fix slice bug;

* remove no need dep