* Reconstruct all API related to lr scheduler, unify dygraph and static

* Reconstruct all API related to lr scheduler, unify dygraph and static

* fix doc

* fix doc

* fix doc of lr_scheduler

* fix unittest and english doc

* fix english doc

* fix confilt

* fix doc

* fix syncbn, test=develop

* fix test,test=develop

* fix unittest,test=develop

* fix RandomSampler & BatchSampler. test=develop

This PR enhance CPU kernel and add new CUDA kernel to make grid_sample support:

- align_corners: with bool type.
- padding mode: which can be in ['zeros', 'reflect', 'border']
- Interpolation mode: which ca be in ['bilinear', 'nearest']

The old CPU and CUDNN version only support align_corners=true, padding_mode='zeros' and interpolation_mode='bilinear'.

The behavior of the new version op in default mode is compatible with the old version.

* find timeout unittests

* setting timeout value

* fix some error

* fix some error

* fix some error

* fix no newline of end file error

* Add MultiHeadAttention api.
test=develop

* Add MultiHeadAttention cache type and gen_cache.
test=develop

* Add TransformerEncoderLayer and TransformerEncoder.
test=develop

* Add Transformer decoder apis.
test=develop

* Add Transformer api.
test=develop

* add unittests for transformer api

* add unittests for transformer api

* Fix some bugs in Transformer apis.
test=develop

* add unittests for encoder, decoder and transformer

* clean conflicts infor in code

* clean Chinese comments

* Add TransformerDecoderCell and TransformerBeamSearchDecoder.
test=develop

* Remove TransformerDecoderCell and TransformerBeamSearchDecoder temporarily.
test=develop

* Add import for Transformer apis.
test=develop

* Update usage of weight_attr and Tensor in Transformer api docs.
test=develop

* Update Transformer apis by renaming MultiheadAttention and cal_kv according to comments.
test=develop

* Fix MultiHeadAttention in test_transformer_api.py.
test=develop
Co-authored-by: NLiuChiaChi <709153940@qq.com>

change to use bce_loss op, add numel check for bce_loss.

* add div, floor_div, remainder

add the isnan, isfinite, isinf api for the paddle 2.0

* add alpha_dropout in nn.functional and nn.layer, test=develop

* refine Interface and assertion, test=develop

* fix ci import error, test=develop

* fix alias and use layers.scale, test=develop

* fix doc and training params, test=develop

* refine details in doc, test=develop

* fix doc details, test=develop

refine Optimizer/Adam/Admax/RMSProp && add Admw

* buf fix

* update comment

* unify arguments place; notest

* fix ut, test=develop

* bug fix

* fix conflicts, test=develop

* add examples code

* bug fix

* fix comments

* fix sample code

* add sample code for Optimizer

* add adamax ut, test=develop

* fix rmsprop ut, test=develop

* add ut for optimizer.py and adamw.py

* remove TestAdamOptimizerBetaVariable

* update api && add ut

* update doc && fix ut

* add ut
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* update Conv2d Conv3d conv2d conv3d api

* add the new api and op for the argmax, argmin

* update conv_transpose api

* add static data

* solve the initial configuration about fleet and rolemaker
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* add binary cross entropy with logit loss

add paddle.tensor.linalg.diag API, diag_v2 OP and CUDA kernel.

* add matmul_v2

* [API2.0] add dropout, dropout2d and dropout3d, test=develop

* refine Interface and assertion after review, test=develop

* fix alias p=1 and use scale, test=develop

* fix doc and training, test=develop

* fix doc in Dropout2D, test=develop

* fix the comment of scatter

* add bernoulli op

* fix cuda kernel and add unit test

* refine doc

* fix uniform

* add functional ctc_loss and CTCLoss class.

* modified docstring of ctc_loss and CTCLoss

* add error message