replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum (#54152)

* replace the AdadeltaOptimizer with Adadelta

* replace the RMSPropOptimizer with RMSProp

* replace the LambOptimizer with lamb

* replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py

* fix bug

* fix bug

* fix bug

* fix bug of Lamp

* fix bug of Lamp

* fix bug of import

* replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer

* fix bug

* fix bug

* Update optimizer.py

* fix bug

* fix bug

* remove function sequence_conv_pool

* fix test_glu.py

* remove function scaled_dot_product_attention

* remove function img_conv_group

* remove function simple_img_conv_pool

* delete the nets.py

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* fix bug

* Update test_image_classification.py

* Update test_recognize_digits.py

* Update test_recommender_system.py

* Update test_image_classification_fp16.py

* fix bug

* fix bug

* fix bug

* remove the nets.py in fluid

* fix bug

* fix the codestyle

* fix conflict

* fix bug

* Fix TODO

* support promote dtype for static amp training

* unify o1 and o2

* update for unittest

* fix op_role

* add use_promote arg

* fix doc

* add promote unittest

* polish unittests

* fix controflow and test

* autofix
Co-authored-by: NLiyulingyue <83450930+Liyulingyue@users.noreply.github.com>

* revert changes in python/paddle/distributed/fleet/utils/hybrid_parallel_util.py

* empty commit, trigger ci

* fix test_slice

---------
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* update

* update

* remove transpiler

* Revert "remove transpiler"

This reverts commit 46044ccd52011d45d7026786d331f264a6a8f645.

* Revert "Revert "remove transpiler""

This reverts commit 80ad0945401b5b5efebac4baee0ec50a793d4405.

* codestyle

* fix setup

* fix

* fix

* replace paddle.fluid.layers.data and remove io.data

* partial commit

* partial commit

* partial commit

* partial commit

* partial commit

* partial commit

* remove data in fluid.layers.io.__all__

* fix errors

* fix unitests

* fix unitest

* fix unitests

* fix unitest

* fix unitest

* fix unitests

* fix unitest

* fix test_layers unitests

* fix typro

* fix unitest

* fix unitest

* fix unitest

* fix typro

* fix unitest test_model_cast_to_bf16

* fix test_reducescatter

* fix collective unitest

* fix collective unitests

* fix collective unitests

* add coverage

* fix add layers.data

* re run ci

* fix some typro

* fix samplecode error

* fix samplecode error

* move fc from fluid to paddle2.0

* fix unit test

* fix some examples

* fix some examples

* move conv2d

* code-style and build bug

* restore c++

* code-style

* rm fluid init conv2d

* remove nn.conv2d

* modify doc

* remove dropout

* remove pool2d

* modify slice infershape

* code style

* modify slice_unittest

* temp fix

* batch_norm api move

* code_style

* codestyle

* ci_static

* add __init__

* reset other change

* revert .cc

* add import batchnorm

* conflict and revert

* fix bug

* fix third conflict one day

* fix conflict

* fix conflict bug

* fix conflict bug

* modify api

* code_style

* modify doc

* add lost doc stable

* fix conflict bug

* ci lack of gpu

* clean elem_arithmetic part2 unittest

* fix test_model_cast_to_bf16

* restore test_model_cast_to_bf16

* clear fluid apis in fleet and passes

* fix model.py

* fix model.py

* fix cpp_pass.py

* clear loss.py

* change test file

* fix some test_*.py

* fix adaround.py

* fix evaluator.py

* fix CI bug

* fix CI bug

* fix decode.py

* fix detection.py

* fix ci bug

* rm test_sigmoid_cross_entropy_with_logits_op_ipu.py and fix __init__.py

* fix ci bug

* fix ci BUG

* mv accuracy and auc

* rm distributions

* rm get_places

* replace metric

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* [CodeStyle] remove all future import

* revert test_error.py

* restore future import in example code

* change fluid.mean to paddle.mean

* reverse some old code examples

* tempfile_fix

* update

* fix_CI

* update_word2vec.inference.model

* remove_change_in_word2vec_book

* fix_word2vec_book

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* clip op extra information when export model,test=ocr

* rename clip_extra parameter to kwargs in save_inference_model, test=ocr

* add static.amp into setup.pu.in

* add unittest for api

* skip reader op in mixed_precision decorator

* add ut

* default open dygraph mode

* fix CI-Mac

* fix Mac-CI other unittest file

* fix CI-Py3

* fix test_communicator_geo and test_buffer_shared_memory_reuse_pass

* add enable_static to fix CI-Py3

* add enable_static to fix CI-coverage

* delete try except

* add custom black varname test=develop

* fix dtype test=develop

* fix num test=develop

* fix ut test=develop

* fix coverage test=develop

* fix blackvar names test=develop

Optimize amp for multi-gpu to enable FP16 gradients transfer across gpus

Fix float16 optimizer

init new amp, optimize inserting cast op for batchnorm

* remove unused inference_transpiler unit-tests

test=develop

* remove InferenceTranspiler usage in quantize_transpiler.py

test=develop

* Add LAMB optimizer

* Expose LAMB Optimizer's APIs

test=develop, test=document_preview

* Cleanup code & doc

test=develop, test=document_preview

* Update lamb optimizer's formula

test=develop

* init auto loss scaling

test=develop

* change API.spec

* change ifelse to switch and use reduce_sum to optimize checking isfinite

test=develop

* Remove redundant code

test=develop

* Init mixed precision training interface

* Add fp16 test script

test=develop

* All initializers support float16

test=develop

* Code cleanup & add more code annotations

test=develop

* Update API spec

test=develop

* Add usage example in doc

test=develop

test=develop

This reverts commit f5d5d7b2.