* add rmsnorm kernel
* add static graph test
* fix round type
* use alignas to avoid msvc compile error
* remove redundant headerfile to avoid rocm compile error
* fix rocm compile not found cub
* Add document

* fix prim add fill_any_like bug

* polish code

* style

* more

* update ctest

* Update legacy_backward.yaml

* Update legacy_ops.yaml

* Update legacy_ops.yaml

* update

* update

* update for move

* fix api rename

* support amp logic for some prim ops

---------
Co-authored-by: Nkangguangli <kangguangli@hotmail.com>

* move batch_norm prim test to op_test

* fix optest bug

* add test to cmake

* add cinn test case

* fix batch_norm prim grad bf16

* fix code

* add cuda check

* fix batch_norm bfloat16

* fix cpu bfloat16 bug

* skip non-bfloat16-supported platform

* fix code

* fix cinn rtol and atol in bfloat16

* fix name

* fix config

* bump ruff to 0.0.271 and update config

* exclude third_party

* bump ruff to 0.0.272

* refine config

* support layer_norm prim op bf16 dtype

* polish code

* resolve conflict

* support activation prim op bf16 dtype

* remove useless code

* fix the strong Wolfe conditions for line search

* pass the max_line_search_iters parameter into line search method (strong_wolfe)

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* fixed cyclic reference that caused patial import

* fixed bad change

* fix bad import

* fix bad import

* fix bad import

* fix ut failed caused by change in_dynamic_mode

* fix ut failed caused by change in_dynamic_mode

* fixed usage of in_dynamic_mode() or in_dygraph_mode()

* revert python3 to python in .pre-commit-config.yaml

* fix merge conflicts

* Reorganize the forward codes of flash-attention.

* Fix forward.

* Remove some noused codes.

* Simplify codes and fix backward.

* Change all LOG(INFO) to VLOG and fix the backward.

* add scale for AF2 flash_attn, much thanks to xreki and shaojie for debug these codes

* decrease the effect of debug print on performance

* Unify the initialize of flashattn arguments.

* Rewirte the reshape of temp_mask and temp_bias.

* API support use_flash_attn.

* Fix compiling error on CI.

* Try to crop the flash-attention lib.

* Correct the condition of whether can use flash-attn.

* Remove the softmax_out argument.

* Remove is_causal.

* Polish codes.

* Fix qkv_transpose_out's shape and scaling of Q * K.

* Update commit of flash-attention.

---------
Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* Fused elementwises kernels and ops

* change fuse pass name

* adjust .pbtxt files

* adjust quantization attributes

* add missing arguments and fix others, review fixed

* simplify fused kernel registration

* fix elementwise unit tests

* reuse one fused elementwise op

* adjust proto

* Add supported datatypes

* Change 'Scale' to 'scale' in tests, change some tests to onednn

* Revert breaking changes

* Fix unit tests

* Delete obsolete test cases

* Delete commented out code

* Fix codestyle

* delete temporary condition

* fix conflicts and delete duplicate fusing

* Fix code after merge

* Move tests to new directory

* fix tests volatility

* Rename test_elementwise_add_onednn_op.py to test_elementwise_add_mkldnn_op.py

* Update CMakeLists.txt add mkldnn op test

---------
Co-authored-by: NSilv3S <slawomir.siwek@intel.com>

* move DataLoader to paddle.io. test=develop

* Add fused_gate_attention API.

* Implement FusedDropout API.

* Fix doc and add unittest.

* Skip for non-gpu device.

* Add unittest.

* modify concat_grad add sum comp rule

* modify opcompat

* modify numel in lbfgs and add a new test case. test=develop

* change param 'lr' to 'learning_rate' in lbfgs and its test

* add opt LBFGS and change test

* add leaky relu composite rule

* add public python api

* unset default negative slope

* fix unittest case

* relocate metri_op.py

* reloacte nn.py

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix variable->tensor and fix __all__

* fix ctr_metric_bundle and sparse_embedding

* fix bug of function init

* fix bug of importing sparse_embedding and ctr_metric_bundle

* fix bug

* Update __init__.py

* support 0-D output and 0-D as indice in __getitem__

* fix tests

* fix inference and UT

* add unittest for setitem

* fix xpu test

* fix xpu 0-d

* Revert "remove ASCEND* keyword (#53046)"

This reverts commit 7fa415ca.

* Delete ascend_trigger_op.cc

* revert-53046-remove_ASCEND_keyword

* update

* update

* remove ASCEND* keyword

* update docstring

* bug fixed

* bug fixed

* fix graph_reindex

* fix

* Update op_compat.yaml

* add random control for fused dropout add

* add __init__

1. modify set_value op, use Scalars to represent attr `values`, instead of a bunch of attributs of various types; (#52408)

2. add program converter and set_value op as an example, which provides the functionality to convert `paddle::framework::ProgramDesc` between old and new formats(the differences are mainly some operators with incompatible updates in the definition);
3. program version and operator version map now are always saved when serializing `paddle::framework::ProgramDesc` to identify the version;
3. provide an option `legacy_format=false` in  serialization of `paddle::framework::ProgramDesc`, it decided whether to convert ProgramDesc back to a legacy format, which is compatible for paddle 2.4.2 or earlier versions to load and execute;
4. deserialization of `paddle::framework::ProgramDesc` is now automatically detecting whether the bytes it receives is in legacy format(contains any of the operators that has been incompatibly updated and have any attribute of type `Scalar`) and convert it to new format. But if you want a faithful deserialization without the automatic conversion, you can use protobuf's deserialization instead. Though it is not recommended, it can be used for the purpose of testing.

* add more infer var type

* fix split error

* fix ut

* fix top_k infer vartype

* fix top_k infer vartype