* fix syncbn convet

* add unittest

* Improve performance of elementwise_add grad op (#29187)

* pass stop_gradient for cast op

* improve performance of elementwise_add grad

* use tensor copy async

* dygraph branch

* fix dygraph branch

* add ut

* make gelu fp16 computing more robust (#29484)

* Add fast path for dropout when p == 0  (#29553)

* add fast path for p == 0 in dropout

* add ut

* Layer norm fp16 (#29169)

* add fp16 for layer_norm op

* revert layernorm api

* fix forward

* fix forward

* fix backward for layernorm with fp16

* fix unit test for layernorm with fp16

* fix with_mkldnn compile error for layernorm with fp16

* 1. revert to PADDLE_ENFORCE_NOT_NULL, 2. change static_cast<float> to static_cast<U>

* fix with_mkldnn compile error for layernorm with fp16

* fix with_mkldnn compile error for layernorm with fp16
Co-authored-by: Nzhiqiu <chenqiuliang@baidu.com>

* fix layer_norm accuracy (#29434)

* Layernorm opt (#29522)

* layernorm fw opt

* layernorm bw opt

* fix typo, test=develop

* remove const dim3 for windows CI compatibility

* merge develop
Co-authored-by: Nzlsh80826 <zlsh80826@gmail.com>

* Fix compile problem when cuda_arch < 6000 (#29576)

* fix compile problem when cuda_arch < 6000

* refine code

* refine code
Co-authored-by: Nzhiqiu <chenqiuliang@baidu.com>
Co-authored-by: Nzlsh80826 <zlsh80826@gmail.com>

Change-Id: Ia5279b0cbb6a5b3970aff66e9510e0d85efa70ce

* fix bn docs (#30096)

* add attribute for batch_norm (#29950)

* add attribute for batch_norm

* add fp16 check into max and avg pool (#29479)

* Add ReserveSpace in dygraph batch_norm. (#29221)

* Add ReserveSpace in dygraph batch_norm.

* Add unittest for reservespace

* add float16 into adaptive_avg_pool2d check list. (#29547)

test=develop

* add alias for upsample, test=develop

* add alias for upsample

* fix example

* fix doc bugs test=document_fix

* fix code bugs test=document_fix

* fix code bugs test=document_fix

* fix doc bugs test=document_fix

* fix doc bugs test=document_fix

* fix doc bugs test=document_fix

* fix conv_transpose bug when padding=same

* [cherry-pick] cherry-pick of PR#29928

* delete paddle.metric.chunk_eval and paddle.metric.mean_iou

* delete paddle.nn.clip and paddle.nn.clip_by_norm

* delete paddle.nn.functional.activation.hard_sigmoid and paddle.nn.functional.activation.hard_swish

* [cherry-pick] cherry-pick of PR#29928

* fix extension import error

* fix wmt14 doc, remove backward, add bidirect direction in rnn api

* fix rnn unittest

* fix test_rnn_nets_static.py bug

* Fix api docs in RNN, Transformer, layer_norm, WeightNormParamAttr.
test=develop

* Fix api doc for print in label_smooth.
test=develop

* Update api docs according to review comments.
Add name argument in RNN back.
test=develop

* fix gpu emb out of range

Change-Id: I5794ac73bd634d5ea069a6fbbd914274b6d6b7bf

* fix doc

Change-Id: I5a3350b2930a9ab2f52116c192b087307faf8fdf

* fix multiple documentation error, test=document_fix

* fix more rst syntax errors, test=document_fix

* fix format issues in docstring, test=document_fix

* move temporal_shift to functional

fix nll_loss doc;test=document_fix

* fix en doc, test=document_fix

* add blank after code declare, test=document_fix

* refine doc of dropout, test=document_fix

* refine npair_loss and dropout, test=document_fix

* fix 2.0 api, test=develop

* fix api, test=develop

* fix doc example, test=develop, test=document_fix

* fix interpolate example, test=develop;test=document_fix

* fix format, test=develop, test=document_fix

* update upsample doc, test=develop, test=document_fix

* 1. grid_sample
1.1 fix has_print
2. conv1d_transpose
2.1 fix code_example error
3. conv1d
4. affine_grid
4.1 has_print
4.2 has_disable_static
5. Conv1DTranspose
5.1 fix code_example error
5.2 has_disable_static
6. Conv1d
6.1 code_example
6.2 has_disable_static

Fix ops doc for some ops 

* fix examples, test=document_fix

* fix, test=document_fix

* fix syncbn

* add unittest

* rewrite the sigmoid_focal_loss code example. test=develop

* fix spelling mistake in comments of code example.test=develop

* change print([.*].numpy()) to print([.*]) in example codes of sigmoid_focal_loss. test=document_fix

* save one name in cross_entropy and softmax_cross_entropy, test=develop

* change used function in CrossEntropy from softmax_cross_entropy to cross_entropy, test=develop

As the title

* add soft_label and axis for CrossEntropyLoss and improve performance,test=develop

* fix conflict in nn/functional/loss.py, test=develop