* modify numel in lbfgs and add a new test case. test=develop

* change param 'lr' to 'learning_rate' in lbfgs and its test

* add opt LBFGS and change test

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* add lamb optimizer and unittest

* fix lamb

* fix lamb v2 op

* fix sampling id

* fix lamb sample code

* Update lamb.py

* fix doc

* fix doc

* Update lamb.py

* modify cond while_loop to paddle.static.nn.cond

* modify crop_tensor to paddle.crop

* modify Variable to paddle.static.Variable

* remove nn.beam_search, nn.beam_search_decode, nn.gather_tree

* remove bpr_loss, center_loss, rank_loss, smooth_l1, teacher_student_sigmoid_loss, edit_distance, sampled_softmax_with_cross_entropy in nn.functional

* remove apis in nn.functional.learn_rate.py

* remove pool2d, pool3d, adaptive_pool2d, adaptive_pool3d in nn.functional

* remove apis in nn.functional.vision

* remove erf, soft_relu in nn.functional.activation

* remove apis in nn.functional.extension

* remove nn.functional.rnn

* remove hash from nn.functional.lod

* remove row_conv from nn.functional.extension

* remove one_hot, pad2d, pad_constant_like from nn.functional.common

* remove nn.gather_tree, nn.BilinearTensorProduct, nn.Pool2D, nn.Pad2D

* remove apis from optimizer.__init

* remove tensor.creation.fill_constant

* remove elementwise_mul in nn.functional.common and  modify to paddle.multiply

* remove  tensor.stat.reduce_mean

* remove reduce_all, reduce_any in tensor.logic

* remove apis in tensor.math

* remove apis in tensor.__init__

* remove has_inf, has_nan in tensor.search

* remove apis in framework.__init__

* remove apis in paddle.__init__

* remove apis in nn.functional.__init__

* modify removed alias apis to raw api in doc and unittests

* fix remove grid_sample bug

* modify removed alias apis to raw api in doc and unittests

* modify removed alias apis to raw api in doc and unittests

* modify removed alias apis to raw api in doc and unittests

* modify removed alias apis to raw api in doc and unittests

* modify removed alias apis to raw api in doc and unittests

* modify removed alias apis to raw api in doc and unittests

* delete alias api relastions in doc

* reserve paddle.compat, paddle.sysconfig

* remove unittest for paddle.reduce_all, paddle.reduce_any

* modify removed alias apis to raw api in doc and unittests

* recover paddle.save and paddle.load

* resolve conflicts

* fix sample code missing paddle.enable_static() bug

* fix sample code missing paddle.enable_static() bug

* fix to_string sample code error

* fix norm api doc, test=develop

* fix error message, test=develop

* fix api norm, test=develop

* add adagrad, test=develop

* fix bug, test=develop

* fix bug, test=develop

* add spetral_norm, test=develop

* fix adagrad, test=develop

* merge , test=develop

* fix doc and unittest of 2.0 lr_scheduler

* fix doc of 2.0 lr_scheduler

* fix unittest

* fix english doc of lr_scheduler

* fix api name of lr scheduler

* fix api name of lr scheduler

* test=develop, update example

* add doc; notest

* fix doc; notest

* update doc; notest

* refine optimizer && adam

* refine optimizer; notest

* add adam

* fix doc

* fix doc && add adamw; notest

* add error message

* bug fix

* refine rmsprop && adamax

* fix ci

* buf fix

* update comment

* unify arguments place; notest

* fix ut, test=develop

* bug fix

* fix conflicts, test=develop

* add examples code

* bug fix

* fix comments

* fix sample code

* add sample code for Optimizer

* add adamax ut, test=develop

* fix rmsprop ut, test=develop

* add ut for optimizer.py and adamw.py

* first commit of adadelta optimizer

* fix learning rate

* fix adadelta doc and add sgd momentum

* remove unused fluid

* fix codestyle

* Update test_adam_op.py

* Update test_adam_op.py

* fix SGD in 2 unittests

* fix SGD in 2 unittests

* fix ci

* fix ut
Co-authored-by: NMRXLT <xlt2024@gmail.com>
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* Reconstruct all API related to lr scheduler, unify dygraph and static

* Reconstruct all API related to lr scheduler, unify dygraph and static

* fix doc

* fix doc

* fix doc of lr_scheduler

* fix unittest and english doc

* fix english doc

* fix confilt

* fix doc

refine Optimizer/Adam/Admax/RMSProp && add Admw

* buf fix

* update comment

* unify arguments place; notest

* fix ut, test=develop

* bug fix

* fix conflicts, test=develop

* add examples code

* bug fix

* fix comments

* fix sample code

* add sample code for Optimizer

* add adamax ut, test=develop

* fix rmsprop ut, test=develop

* add ut for optimizer.py and adamw.py

* remove TestAdamOptimizerBetaVariable

* update api && add ut

* update doc && fix ut

* add ut
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* add base name alias; test=develop

* fix embedding bug; test=develop

* fix io error; test=develop

* add metric; test=develop

* add import palce; test=develop

* add setup; test=develop

* fix incubate reader; test=develop

* fix initializer error; test=develop