* add mish operator and api

* remove redundant code and modify grad_atol of mish unittest

* modify mish code to be consistent with other activation implementation

* add fold opereators, test=develop

* add fold opereators, test=develop

* add fold opereators, test=develop

* update fold op error test, test=develop

* fix unitext, test=develop

* fix unitext, test=develop

* add hinge_embedding_loss

* fix test_API

* test_API succeed

* add English doc

* fixed using of expired fluid api

* fix doc

* fix doc and rm python/paddle/fluid/layers/loss.py

* get raw python/paddle/fluid/layers/loss.py back

* fix Examples bug in English doc

* unique -> flatten

* fix api code

* fix English doc

* fix functional loss English doc

* fix Example doc

* .numpy() -> paddle.unique()

* fix unique

* fix label_item_set

* modified judgment equation

* Got a beautiful loss equation

* use paddle.to_tensor

* fix loss and add static check

* fix loss and add static check

* delta -> margin

* promote to v2

* alter

* add maxunpool2d in __all__

* fix MaxUnPool2D example

* support data_format='NHWC' for prelu channel mode

* add zeropad2d v0.1

* add zeropad2d v0.2

* add zeropad2d v0.3

* add zeropad2d v0.3

* add zeropad2d v0.3

* add zeropad2d v0.4

* add zeropad2d v0.5

* add zeropad2d v0.5 codestyle

* add zeropad2d v0.5 codestyle

* add zeropad2d v0.6 functional

* add zeropad2d v0.6 functional

* add zeropad2d v0.6 functional

* Revert "Align CTC grad scale same with ESPNet (#34729)"

This reverts commit 10f9644c.

* ctc grad compute on gpu

功能：本PR的目标是提高attention模块的计算性能。
为了减少框架层对op的调度开销，本PR通过在C++层手动实现attention模块，对外提供attention 大op；
为了减少防存开销，本PR采取了两种优化方法：
（1）在q,k,v计算时通过共享输入X，将该处的gemm，transpose和bias add从三次调用减少为一次；
（2）使用kernel融合优化技术，在不同cuda kernel之间通过寄存器传输数据；

功能：本PR的目标是提高attention模块的计算性能。
为了减少框架层对op的调度开销，本PR通过在C++层手动实现attention模块，对外提供attention 大op；
为了减少防存开销，本PR采取了两种优化方法：
（1）在q,k,v计算时通过共享输入X，将该处的gemm，transpose和bias add从三次调用减少为一次；
（2）使用kernel融合优化技术，在不同cuda kernel之间通过寄存器传输数据；

* fix BatchNorm for fp16

* update

* update

* update

* try make CI pass

* doc typo

* update doc string

* fix unpool doc, test=document_fix

* fix typo for python example, test=document_fix

* Change the invoking method of settiem from numpy to set_value op when value is not tensor

* fix the check logic for inplace in setitem

* fix the unittest problem caused by setitem doesn't support fp16

* modify some code format in setitem

This PR adds error exception in fused transformer python interface.
The function body are not implemented (will be implemented later).
Following zhiqiu's comment in previous PR-35206 (merged already), it is better to raise an exception instead of using "pass".

This PR adds fused transformer python related files. It defines interface of fused transformer.

Fused transformer implements an optimized version of transformer layer (in python/paddle/nn/layer/transformer.py). In this PR, four layers (functions) are defined:
(1) FusedMultiHeadAttention: multi-head attention layer
(2) FusedFeedForward: feed forward layer
(3) FusedTransformerEncoderLayer: transformer encoder layer
(4) FusedTransformer: transformer layer

* add maxunppol2d op, test=develop

* fix typo, test=develop

* fix unpool unitest, test=develop

* fix unpool code-example, test=develop

* fix for unpool_op_unittest,test=develop

* fix example code, test=develop

* add noqa:F401, test=develop

* fix converage, test=develop

* fix unitest for unpool, test=develop

* rename unpool2d to unpool, test=develop

* rename unpool2d to unpool, test=develop

* add (N,C,*) input support for GroupNorm

* --amend

* fix pad outliers err

* fix pad api input type and doc

* fix example of pad

* add unittest for pad3d

* fix unittest

* fix error format

* fix pad doc

* dygraph support more ctc grad scale

* scale for 1.x

* fix unitest

* fix unitest

* format code

* fix unittest

* fix log info

* unittest cov

* fix format;notest,test=cpu,coverage

* skip ctc_loss egs;test=cpu

* warpctc grad cov;test=coverage

* add dygraph test;test=coverage

* format;test=cpu,coverage

* format;test=cpu

* add api compat;test=cpu

* add cpu test

* rename

* rename

* fix

* fix test

* format

* eigen cpu

* eigen gpu grad pass

* cuda gpu pass

* format

* fix ci

* fix paddle.optimizer test=document_fix

* fix paddle.optimizer test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* test=develop

* update identity

* add unittest

* notest,test=mac_py3

* modify comment & testname

* test=document_fix

* update comment

* test=document_fix

* activate all of the CI

* fix the order of unfold parameters

* cache core.ops, test=develop

* refine, test=develop

* fix output padding conv

* add repr unittest for conv

* fix syncbn

* improve group norm cpu precision and performance

* add unit test to group norm

add __all__=[] to python files not in API public list; import * only support in API public list files (#32643)

* update 2.0 public api in nn

* replace Chinese character cause error in ci;synchronization with pr:#32588 to avoid 'ascii' codec in python2

* numbers used in paddle.nn.functional.norm but not imported

* fixed docs.

* Fixed docs. test=document_fix

code bak.

fixed docs. test=document_fix

* Revert to previous version of python/paddle/fluid/backward.py

* fixed bugs.

* test=document_fix. Fixed examples.

* fix bn docs

* fix unittest

* Modified params of some APIs to support tuple and list.

* fixed bug.