* cherry-pick fix dataloader early reset bug, test=release/1.7

* cherry-pick implementation changing, test=release/1.7

* fix ut, test=release/1.7

test=release/1.7, test=develop

* add learning rate api; test=develop

* fix uni test converage; test=develop

* fix travis ci error; test=develop

* fix comment; test=develop

* fix example error; test=develop

* polish the api description, test=develop
Co-authored-by: Nzhongpu <2013000149@qq.com>
Co-authored-by: Nhong <43953930+phlrain@users.noreply.github.com>

* add parameter check; test=develop

* change parameter name checker in dygraph guard; test=develop

* fix test layers error; test=develop

* revert some code to develop; test=develop

* fix exampel error; test=develop

* fix comment error; test=develop

* fix comment error; test=develop

* cherry-pick #22266

* cherry-pick #22169
test=release/1.7, test=develop

* test=develop, bug fix for sparse recorder

* add NotImplementedError for multi optimizers used on multi-places . test=develop

* assert error only if num_devices>1. test=develop

* set test_optimizer_in_control_flow in CMakeLists for using multi-GPU.test=develop

* test Optimizer in dygraph, test=develop

* add optest for Optimizer in dygraph, test=develop

* fix adagrad optimizer, test=develop

* fix dpsgd optimizer, test=develop

* fix test_optimizer.py, test=develop

* fix dpsgd optimizer, this op only support cpu, test=develop

* add optest for optimizer, test=develop

* add description for dpsgd, test=develop

* add rmsprop to white_list in unused_var_check.cc, test=develop

* polish code style, test=develop

* polish code style, test=develop

* delete seed attribute for DpsgdOptimizer, test=develop

* change testing to debugging, test=develop

* use snake_cased Layer name

* add named_parameters and named_sublayers api

* add include_sublayers param, add unittest
test=develop

* fix named unittests
test=develop

* fix unittest
test=develop

* add api docs
test=develop

* arg fix
test=develop

* reserve rnn_impl name_scope for static graph
test=develop

* fix load static param
test=develop

* fix load static param
test=develop

* add clear_gradients for Optimizer, add api description, test=develop

* fix optest for optimizer's clear_gradient interface, test=develop

* add sample code, test=develop

* polish sample code, test=develop

* update layers, test=develop

* update layers for resnet, test=develop

* fix is_test attr, test=develop

* update cycle_gan, test=develop

* update reinforcement_learning, test=develop

* update ocr, test=develop

* fix bug, test=develop

* update PRelu arg
test=develop

* fix unittests
test=develop

* fix element mode alpha shape
test=develop

* split channel_or_input_shape arg
test=develop

* fix unittest
test=develop

* modify fc to linear in sample code, test=develop

* remove FC, test=develop

* remove warnings, test=develop

* drop fluid/imperative/README.md , test=develop

* change fc to linear, test=develop

* polish code style, test=develop

1. Add a new input named batch_roi_nums for prroi_pool_op. batch_roi_nums includes the number of roi for each image in batch when rois is Tensor. This information is saved in rois's lod when rois is LoDTensor.
2. add grad check to prroi_pool_op and solve unnormal X grad diff in CPU.

* modify test framework, test=develop

* fix test_fsp_op, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* add check for threshold, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* up, test=develop

* update, test=develop

* update, test=develop

* update, test=develop

enhanced ops: acos, asin, brelu, ceil, cos, elu, floor, hard_shrink,
hard_sigmoid, hard_swish, relu6, round, rsqrt, sin, softshrink,
cos_sim, cross_entropy, im2squence, kldiv_loss

* enhance load interface; test=develop

* add uni test and add comment; test=develop

* fix converage; test=develop

* use path.joint replace "/"; test=develop

* windows debug; test=develop

* fix window unitest error; test=develop

* fix commet error; test=develop

* add model shuffix check; test=develop

* fix example error; test=develop

* support dygraph structured name; test=develop

* add load static param unitest and fix save load; test=develop

* fix varBase import error; test=develop

* fix unitest error; test=develop

* add comment for parameter; test=develop

* fix uni test error; test=develop

* change parallel se-resnet; test=develop

* fix dygraph se resnext parallel test; test=develop

* remove useless code; test=develop

* remove useless code; test=develop

* set esp as 1e-6 to solve elu unitest fail,test=develop

* add special way to add distribute vars， Update Pyramid hash op

* Fix the global_step & continuous applying error in EMA

test=develop

* Fix for step 0 & add unit test, test=develop

* fix grad clip， clip op belongs to Backward op when running in Parameter Server mode.

* add erf op and python interface.

* add fp16 support for erf op.

* add unitests for erf op and its python interface.

enhanced ops: conv2d, conv3d
elementwise_pow: change to a reasonable shape

* support elu activation double grad,test=develop

* delete the code commit in .cc,test=develop

* fix relu test unpass, test=develop

* add elu double grad kernel and unit test

* add caculate dX in elu double grad functor, test=develop

* update the commit code,test=develop

* add dygraph LayerList

* add unittest
test=develop

* add newline
test=develop

* typo fix
test=develop