* move worker loop to top level

* move reader process loop to top level

* fix failed unittests

Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for static graph amp training. (#26240)

* update amp_check_finite_and_scale_op for static_amp.

* use amp_check_finite_and_scale in static graph amp.

* update grads to zero when grads own infinite values(as for amp_checkout_finite_and_scale op).

* add update_loss_scaling op in cpp.

* add update_loss_scaling_op unit test.

* update the doc of the check_finite_and_unscale op

* Update the process of gradients updating skipping if the gradients have infinite values.

* update the way to zero grads.

* update test_update_loss_scaling_op.py

* add log info when find infinite grads.

* add the unit test for UpdateLossScaling Layer.

* rm auto from localsgd

* Add int8 GRU kernel with UTs

* Lint fixes

* More lint fixes

* temporally disable zero_copy

* add test

* follow comments

* fix calcu_gradients

* fix code place

* fix embedding interface usage

* support to_static(model)

* add warning and unittest

* Use the single GPU card to execute the test_fuse_bn_act_pass UT.

* add double grad for tile, test=develop

* add double grad for expand_v2 op, test=develop

* add double grad for expand, test=develop

* add some unittest cases ot verify jit.save, no_test

* add more unittests

* add test with example inputs

* polish implement details

* remove useless blank

* fix fetch random error

* parameter_server_optimizer support auto_strategy

 fix the CudaPinMemory bug for the equal op and add the test case for the equal op

Decrease the number of running iterations to reduce CI time.

CI system shows it decreased the unittest time from about 90 seconds to about 30 seconds

add lars to fleet meta optimizer

* refine launch and distributed repr string for print

* set default value to strategy in distributed_optimizer test=develop

* Add naming rule if not specific InputSpec.name

* fix function name typo

* refine comment

* remove print statement

* fix frobenius_norm error, rm p=0 2-axis support. test=develop

* fix summary rnn

* fix kl_div bug when input shape is [1] and reduction is batchmean

* enhance collect_op for dygraph, test=develop

* enhance detection ops with lod, test=develop

* support none bbox left in generate_proposals, test=develop

* unfiy MultiLevelRoisNum, test=develop

* update core.ops, test=develop

* add op register for new input & output, test=develop

* test=develop, add ps auto

* fix weight

* fix weight and fix doc

* fix embeeding padding idx

* add UT

* fix interval

* fix unsqueeze in dygraph

* add test

* add test

* fix typo for interp_v2,test=develop

* align with torch, test=develop

* add area mode, test=develop

* fix bug, test=develop

* format notes, test=develop

* update for converage, test=develop

* fix bilinear, test=develop

* fix bicubic, test=develop

* fix typo, test=develop

* fix coverage, test=develop

* fix helper.input_dtype, test=develop

* polish notes, test=develop

* polish notes, test=develop

* polish notes, test=develop

* fix 2.0api, test=develop

* fix, test=develop

* add auto parallel L1 implementation
test=develop