* add paddle.optimizer.lr.CyclicLR

* add unittest of CyclicLR

* fix code format

* fix bug

* try

* fix CI-Coverage

* fix ValueError

* fix arguments assgin

* fix code format and retry pulling develop to pass ci

* fix typo

* Refactor

* fix function-redefined in test_lr_scheduler.py

* update

* fix conflict

* update

* gamma->exp_gamma

* polish docs

* fix code-style

* adjust code format again

* change format of __all__ in lr.py

【Hackathon No.17】为 Paddle 新增 paddle.nn.CosineEmbeddingLoss 和 paddle.nn.functional.cosine_embedding_loss API (#41680)

* add cosine embedding loss API

* new version

* new version

* new version

* set label to int32

* new version

* new version-test

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* new version

* aligning to Chinese document

* add name parameter

* activate CI

* fix format error

* unit test code format

* format code

* move_group_norm

* move group norm backward

* fix code format

* modify code according comment

* [Dy2stat]Add RollBack into original dygraph function for @to_static

* fix unittest

* support matmulv2 in multihead

* Polish codes and memory usage for fused_gate_attention.

* Fix wrong reduce_dims in fused_gate_attention when computing gradient of nonbatched_bias.

* optimize dump_to_cpu for dymf

* code clean. test=develop

* fix func. test=develop

* fix code style. test=develop

* fix. test=develop

* add method for post ops

* format code

* change post-ops pattern

* code style

* add use_master_acc_grad

* add ut

* Update Profiler

* make HostEventRecorder templated

* fix bugs of reducer

* update

* update

* [MLU]support cast double type

* [MLU]fix cast test

* Update creation.py

* Update search.py

* Update search.py

* Update xavier.py

* Update xavier.py

* Update pooling.py

* Update pooling.py

* Update pooling.py

* Update search.py

Transpose optimization with assitant of  Chengdu Supercomputing Center and auto_tune operation (#42704)

fix the unittest bug of none grad of margin_cross_entropy when FLAGS_retain_grad_for_all_tensor change default setting (#43241)