* use GpuMemcpy and GpuMemset

* sparse convert kernel support double dispatch by indices dtype

* cudaMemcpyKind->gpuMemcpyKind

* [Dy2Stat]Remove fluid.layers in transformer code

* remove useless return_name_vars

* fix unittest

* fix unittest

* fix unittest

* fix depthwise conv yaml error

* fix depthwise conv double grad error

* change tensor name

* fix unittest bugs

* add unary ops

* move to activation_ops

* Add option for test executor profiler

* Change option for test executor_profiler

* Re-implemented check_finite_and_unscale_op  with newly added xdnn api
*test=kunlun

* Re-implemented check_finite_and_unscale_op  with newly added xdnn api

*test=kunlun

* fix structure infos conflict in static return_list mode. test=develop

* add nproc_per_node for DistributedFusedLamb

* fix nproc_per_node communicator bug

* fix ring_id = 1 init bug

* fix ci

* fix test_parallel_executor_mnist.py

[Dy2Stat]Add apply_optimization for @to_static and remove BreakTransformOptimizer by default (#43320)

* [Dy2Stat]Add apply_optimization for @to_static and remove BreakTransformOptimizer by default

* fix unittest

Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* [Dy2stat]Add RollBack into original dygraph function for @to_static

* fix unittest

* [Dy2Stat]Support deepcopy Net instance after @to_static

* add fetch_list

* fix evaluate log

* tiny fix

* add paddle.optimizer.lr.CyclicLR

* add unittest of CyclicLR

* fix code format

* fix bug

* try

* fix CI-Coverage

* fix ValueError

* fix arguments assgin

* fix code format and retry pulling develop to pass ci

* fix typo

* Refactor

* fix function-redefined in test_lr_scheduler.py

* update

* fix conflict

* update

* gamma->exp_gamma

* polish docs

* fix code-style

* adjust code format again

* change format of __all__ in lr.py

【Hackathon No.17】为 Paddle 新增 paddle.nn.CosineEmbeddingLoss 和 paddle.nn.functional.cosine_embedding_loss API (#41680)

* add cosine embedding loss API

* new version

* new version

* new version

* set label to int32

* new version

* new version-test

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* aligning to Chinese document

* add name parameter

* activate CI

* fix format error

* unit test code format

* format code

* move_group_norm

* move group norm backward

* fix code format

* modify code according comment

* [Dy2stat]Add RollBack into original dygraph function for @to_static

* fix unittest

* Polish codes and memory usage for fused_gate_attention.

* Fix wrong reduce_dims in fused_gate_attention when computing gradient of nonbatched_bias.