* add lars to fleet meta optimizer

* add lamb to proto

* add lamb to fleet meta optimizer

* fixed syntax bug

* fixed syntax bug

* fixed syntax error in lamb, add config setter of lamb in distributed_strategy

* trigger unitest to rerun

* add new unitest func for lamb

* revise unitest for lars and lamb

* revise dgc meta unitest

* revise lars document in distribute_strategy

* revise lars lamb document in distributed_strategy.py

* revise lars lamb document in distributed_strategy.py

* add weight decay exclude logic to lars

* restore optimzier.py

* restore optimizer.py as develop except lars

* add epsilon and exclude fn to distributed_sttrategy

* add lars epsilon

* revise unitest for fleet lars and lamb

* revise lars lamb unitest for CI coverage

* revise lars argument api

* revise lars argument api

* revise lars argument api

* revise api doc of lars

* fix op role

* add sharding save and add_sync_comm_for_test function

* add comm_analyse to utlis

* revise sharding_utils

* add sharding saving unittest

* revise sharding utils for unittest

* update interpolate, test=develop

* fix coverage, test=develop

add two paddle-2.0 apis: paddle.static.io.save_inference_model and paddle.static.io.load_inference_model (#28606)

* add two apis: paddle.static.io.save_inference_model and paddle.static.io.load_inference_mode, which are campatible with paddle.fluid.io.save_inference_model and paddle.fluid.io.load_inference_model respectively.

* add unittest for new save_inference_model and load_inference_model. test=develop

* enhance doc. test=develop

* add paddle.enable_static() to test_inference_model_io.py. test=develop

Fix a bug that used in PaddleGAN model which used `isinstance` in a for loop

* adapt pad const

* fix comment and rm fluid import

* rm stdout

* fix note

* fix output dtyp inconsistent with input

* refine code

fix the gradient bug for the topk v2 

* add WeightedRandomSampler. test=develop

* Add checkpoint to quantize

* Change bfloat16 option

* add mkldnn inplace op version test

* update mkldnn_inplace fuse pass

* update the inplace test

* fix test_multinomial

* fix test_multinomial add 0 prob

* DataLoader support not auto collate batch. test=develop

PaddleSeg uses tuple as parameter of transpose in dygraph code:
https://github.com/PaddlePaddle/PaddleSeg/blob/release/v0.7.0/dygraph/paddleseg/models/danet.py#L152

However, in dy2stat, static code doesn't support the perm as a tuple. This PR fixed it.

* Fix gradients with ignore_idx in softmax_with_cross_entropy.
test=develop

* Fix gradients with ignore_idx in softmax_with_cross_entropy on cpu.
Remove softmax_with_cross_entropy from op_threshold_white_list.
test=develop

* Fix test_softmax_cross_entropy_op.py.
test=develop

I found the unittest failed due to batch size. Maybe the reason is that our CI machine has limited memory. I decreased the batch size.

* update, test=develop

this is a preemptive measure, the unit test was failing occasionally with
cudnn internal error

* add onnx export module, test=develop

* add unit test for paddle.onnx.export

* adjust api & doc

* fix some typo

* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie

As the title

As the title

* fix ut exec timeout notest,test=kunlun

* fix error for executing ut timeout,test=document_fix

* modified timeout value on windows

* fix some error

* exec ut no more than 15s 2

* fix for ut test_inplace_addto_strategy timeout

* disable ut test_parallel_executor_fetch_isolated_var,test=document_fix

* test for limiting ut exec time as 15S

* fix an error caused by cannot find ut

* fix some error

* can not find test_transformer

* fix error caused by ut not run in windows

* fix error caused by Compiler Options

* fix error caused by setting timeout value as 15 in python/paddle/tests/CMakeLists.txt

* setting timeout value to 120s for old ut

* add the timeout value setting

* fix error caused by ut only run in coverage_ci

* add analyzer_transformer_profile_tester

* fix some error

* fix some error

* fix error with inference option

* fix error with inference option setting as ON_INFER

* add some ut to set timeout

* modified some option

* fix error

* fix some timeout error

* fix error

* fix error

* fix timeout for test_analyzer_bfloat16_resnet50

* fix error

* setting timeout properity for some ut

* first pr for new ut timeout as 15S

* remove slelected rows all reduce over height check

* polish unittest

* refine jit.save/load to add support for other method, not only forward

* refine the code based on unit tests

* Add unit test for the code

* Add unit test for the code

* Modify the code according to the unit test

* Delete useless comments, save only one info file, etc.

* remove static_mode_white_list.pyc

* edit the code that generate 'extra_var_info'

* fix random failure

* use two input(x)

* fix model