* update rnn.py

* update common.py

* update rnn.py

* update common.py

* fix CI

* add index select op

* add to op teller

* add trt version control

* delete useless code

* support amp logic for layer_norm and softmax

* fix layer_norm amp

* fix layernorm api and dropout fp16

* fix layernorm api and dropout fp16

* fix bn, ln dtype in float16

* fix dropout fp16

* fix comment

* fix cinn dropout amp error

* add assign composite backward op

* fix log msg

* code style

* fix comp rule

* replace assign with by_pass

* [CINN]Enhance CacheKey hash logic by considering input dtypes (#50557)

* [CINN]Enhance CacheKey hash logic by considering input dtypes

* add unittest

* fix typo

* fix typo

* fix map.at

* fix find

* fix test

* fix cinn cache key structure realize

* using ordered map for attributes

* add test by review advice

---------
Co-authored-by: Njiangcheng <thisjiang@qq.com>

* [prim] enable dygraph_to_static to support custom_vjp

* Pr 50885 (#7)

* [CINN]Enhance CacheKey hash logic by considering input dtypes (#50557)

* [CINN]Enhance CacheKey hash logic by considering input dtypes

* add unittest

* fix typo

* fix typo

* fix map.at

* fix find

* fix test

* fix cinn cache key structure realize

* using ordered map for attributes

* add test by review advice

---------
Co-authored-by: Njiangcheng <thisjiang@qq.com>

* [prim] enable dygraph_to_static to support custom_vjp

* fix code in a dy2static-friendly way.

* [dystatic] add hooker for prim

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>
Co-authored-by: Njiangcheng <thisjiang@qq.com>
Co-authored-by: Ncxxly <chenxx_id@163.com>

* [prim] enable dygraph_to_static to support custom_vjp

* fix cast prim and vjp dtype mapping error bug

* Cxx prim custom vjp (#8)

* [CINN]Enhance CacheKey hash logic by considering input dtypes (#50557)

---------
Co-authored-by: Njiangcheng <thisjiang@qq.com>

* [prim] enable dygraph_to_static to support custom_vjp

* Pr 50885 (#7)

* [CINN]Enhance CacheKey hash logic by considering input dtypes (#50557)

* [CINN]Enhance CacheKey hash logic by considering input dtypes

---------
Co-authored-by: Njiangcheng <thisjiang@qq.com>

* [prim] enable dygraph_to_static to support custom_vjp

* fix code in a dy2static-friendly way.

* [dystatic] add hooker for prim

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>
Co-authored-by: Njiangcheng <thisjiang@qq.com>
Co-authored-by: Ncxxly <chenxx_id@163.com>

* [prim] enable dygraph_to_static to support custom_vjp

* fix cast prim and vjp dtype mapping error bug

* [dy2static-ci] fix dy2static ci errors.

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>
Co-authored-by: Njiangcheng <thisjiang@qq.com>
Co-authored-by: Ncxxly <chenxx_id@163.com>

* [Prim] enable whitelist and blacklist for custom_vjp

* support softmax grad

* remove additional code

* add test back

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>
Co-authored-by: Njiangcheng <thisjiang@qq.com>
Co-authored-by: Ncxxly <chenxx_id@163.com>
Co-authored-by: Nxiongkun <807377414@qq.com>

* add option for setup.py

* add option for setup.py

* add option for setup.py

* add option for setup.py

* add ennv_dict.py and dist/ to .gitignore

* add ennv_dict.py and dist/ to .gitignore

* modify .gitignore

* optimizing setup.py develop command

* add libpaddle.so

* modify setup.py

* add python/paddle/distributed/fleet/.gitignore

* add libpaddle.so to .gitignore

* add *.so to python/paddle/libs/.gitignore

* add new gitignore

* support set_default_dtype bf16

* support float

* feat: add relu composite rule

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, add python api of relu

* feat: add relu composite rule, commit hook

* fix: maximum type error & ban cinn test

* fix: maximum input sequence bugs

* resolve conflicts

* fix: code style bugs

* add: relu fp16 test

* feat: add rsqrt composite rule

* feat: add rsqrt composite rule

* resolve conflicts of composite rule

* fix: delete check eager

* remove unit tests about GraphExecutionOptimizer

* remove test file

* remove parallel_executor related unit tests

* fix CI

* support to_static for SpectralNorm

* support amp logic for layer_norm and softmax

* fix layer_norm amp

* fix layernorm api and dropout fp16

* fix layernorm api and dropout fp16

* fix bn, ln dtype in float16

* fix dropout fp16

* fix comment

* remove with_data_parallel in example code

* revert python/paddle/fluid/data_feeder.py

* fix static.nn.fc api

* support gather test on prim and cinn

* reset timeout for gather

* add pow composite rule

* fix test

* fix unit test

* update test

* fix test

* update

* add fp16 test for divide, matmul, pnorm

* add cumsum fp16 unittest

* fix threshold

* revert cumsum

* fix code-style

* fix according to review

* fix kernel not found

* Delete hardswish_raw op

* fix ut

* refine _found_inf

* add split_with_num composite rule

* add split_with_num composite rule

* add split composite rule

* update

* update test

* update test

* delete split_with_num_grad

* first commit

* fix code bugs in for_loop

* fix bugs in cuLoadAddStridedInputs.

* optimization for LayerNormBackwardComputeGradInput

* add unitest for validating the optimization

* fix windows ci error

* cuda graph support multi-stream for new executor

* fix windows compile error

* delete create_cuda_graph_stream

* wisemax fp16 support

* add bf16 support 4 elementwise_max

* append broadcast 4 op 4 fp16 / bf16

* fix elewise_max ut bf16 numeric delta

* append fp/bf16 uts

* add fp/bf16 uts

* change bf16 uts delta

* fix some issue

* add prim 4 fp16

* fix conv2d filter

* init

* modify