1. Support customize eval function instead of eval program.
2. Fix loading checkpoint in quantization strategy.
3. Support saving eval model when saving a checkpoint.
4. Fix decoder of loading context in PaddleSlim.
5. Fix restoring from the checkpoint of uniform prune strategy.
6. Support saving eval model and infer model during training.
7. Add ‘unitest’ for saving eval model, saving infer model and uniform pruning restoring from the checkpoint.
8. Fix pruning of depthwise_conv_grad op by updating the groups.

* add recompute based checkpoints methods for large batch training
test=develop

* add append_backward_with_forward_recomputation
test=develop

* refine optimizer
test=develop

* update backward and optimizer
test=develop

* make Variable usable
test=develop

* add recompute code

* refine optimizer
test=develop

* refine addup _append_backward_ops_with_checkpoints_
1) for recompute part, just cache the grad_op_desc without appending to block
2) before appending grad_op_desc to backward part, addup_repetitive_vars, remove unused branch
test=develop

* make method private

* add recompute strategy into DistributedStrategy
test=develop

* checkpoint version3
test=develop

* remove some print information
test=develop

* remove unused sumop
test=develop

* try to fix recompute with graph building modules

* add input names to vars should be held

* add memory debug tool

* backup backward

* Fix bugs

* add backward desc for op not in any segments

* add exception info for sub_block

test=develop

* modify code style

test=develop

* modify code style

test=develop

* remove print functions

test=develop

* add API spec

test=develop
test=document_preview

* make Recompute a child class of Optimizer

test=develop
test=document_preview

* add API spec

test=develop
test=document_preview

* modify API spec

test=develop
test=document_preview

* add document for Recompute

test=develop
test=document_preview

* change API doc of Rcompute

test=develop
test=document_preview

* code cleaning

test=develop
test=document_preview

* modify API spec

* fix bugs when segments hold no element

* add testcase for Recompute Optimizer

test=develop
test=document_preview

* add test for apply_gradient, and code cleaning

test=develop
test=document_preview

* add test case for load function

* enable CI

test=develop
test=document

* add test case

test=develop
test=document_preview

* add sample code for 4 function of recompute optimizer

test=develop
test=document_preview

* add mse_loss op

* move tree_conv to fluid.contrib.layers

test=develop

* update API.spec for tree_conv

test=develop

* update tree_conv api to increase unit coverage

test=develop

* unify DataLoader APIs, test=develop

* integrate iterable CPU Dataset, test=develop
add GPU dataset supporting, test=develop

* add unittests for dataset, test=develop

* add more docs to dataloader apis, test=develop, test=document_preview

* refine doc, test=develop

* refine doc again, test=develop

* increase coverage, test=develop

* add instance norm op

1. group_norm support data_layout=NHWC
2. modified doc of group_norm

modified interpolate_op to support tensor attribute

1. the parameter out_shape of image_resize、resize_nearest/bilinear/trilinear can be a list or a 1-D tensor variable. If a list, each element can be an integer or a tensor variable with shape: [1].

2. the parameter scale of above Ops can be a 1-D tensor variable.
modified document of image_resize, resize_nearest, resize_bilinear, resize_trilinear and add some code example.

add crop_tensor op. The main difference with crop is :

1. If the argument shape is a list, each element is an integer or a tensor variable with shape: [1]. This way is suitable for the case that the shape may be changed each iteration.

2. If the argument shape is a variable. Its rank must be 1. In crop op, the rank of shape must be the same as x

offsets can be a list, in which each element is an integer or a tensor variavle with shape: [1].

* add precise roi pooling op test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* test=develop

* detail the description test=develop

* test=develop

* elaborate the doc for return type test=develop

* test=develop

Optimize amp for multi-gpu to enable FP16 gradients transfer across gpus

* strided_slice op basic function test=develop

* test=develop rewrite and fix

* fix bug test=develop

* fix for the PADDLE_ENFORCE usage

* add some unit testw

* fix for the aip  test and copright and fix test=develop

* fix API.spec test=develop

* fix API.spec test=develop

* add axis parameter test=develop

* fix for the build error test=develop

* fix python api  test=develop

* fix the build test=develop

* fix build test=develop

* fix API spec test=develop

* test=develop add some comment and single op test

* fix API spece test=develop

* fix test=develop

* fix test=develop

* fix api test=develop

* fix api test=develop

* fix API.spec test=develop

* fix typo test=develop

* fix API.spec test=develop

* fix API typo test=develop

* fix doc and API.spec test=develop

* add deformable conv v1 op, test=develop

improve pow op according to reviews:
1. Delete unnecessary judgement statements in PowGradOpDescMaker;
2. Improve test of test_api;

overload GetKernelTypeForVar

add stop_gradient=True when attr(factor) is tensor Variable, change examples in API pow.
test=develop,test=document_preview

add support parameter inference when argument shape is a list containing integer and tensor variable;
test=develop

fix reshape op according to reviews:
1. improve or message;
2. improve test of test_api.
test=develop,test=document_preview

fix reshape op: Add error message in nn.py, test=develop

add stop_gradient=True when attr(shape) is tensor Variable.
change examples in API reshape.
test=develop,test=document_preview

add support parameter inference when arguments starts or ends is a list containing integer and tensor variable;
test=develop,test=document_preview

improve slice op according to review(from hongyu). test=develop

fix slice op according to review: infer_flags, test=develop

fix slice op: improve overload operator __getitem__ to support attrs(starts and ends) are Variable.
test=develop,test=document_preview

fix test_slice_op: add TestSliceOp_decs_dim_6 to resolve conflict with test_slice_ngraph_op. test=develop

add stop_gradient=True when attr(starts) or attr(ends) is tensor Variable.
test=develop,test=document_preview

1. add tensor support for argument expand_times in expand op;
2. add support parameter inference when argument expand_times is a list containing integer and tensor variable;

improve expand op according to reviews:
1. add doc of ExpandTimes in expand_op.cc;
2. improve the test of test_api.

add stop_gradient=True when attr(expand_times) is tensor Variable, change code examples.
test=develop,test=document_preview

* add one_hot_v2_op to remove last_dims==1 test=develop

* add api unittest code for CI_Coverage test=develop

* improve CI_Coverage rate by adding test_with_depth test=develop

* update dygraph api-doc and backward api-doc, test=develop

* update dygraph api-doc and backward api-doc, update api.spec, test=develop

* update dygraph api-doc and backward api-doc, update api.spec, test=develop

* update API.spec, test=develop

remove unused accuracy-diff warpctc-cudnn implementation

test=develop

* merge_empty_lod_tensor, test=develop

* fix multiclass_nms, test=develop

* refine API.spec, test=develop

* add unittest case for fetch, test=develop

* add lod tensor test, test=develop

* return index for multiclass_nms, test=develop

* add api for multiclass_nms2

* update API.spc, test=develop

* refine api doc, test=develop

* fix test_detection.py, test=develop

* polish code, test=develop

* add more unittest case, test=develop

Fix float16 optimizer

* add prune when test_program.clone is executed after optimizer.minimize

* add unittest, test=develop

* add resnet and transformer test case, test=develop

* add regularization for optimizer & program compare function, test=develop

* add lstm unittest, test=develop

* polish code based on review comment, test=develop

* adapt to interface change in framework._prune, test=develop

* update API.spec, test=develop

* delete args on linear_chain_crf_op doc

* delete args on linear_chain_crf_op doc

* delete args on linear_chain_crf_op doc

* add code example

* fix api doc

* fix doc of crf

* fix doc of crf

* add test=develop

* modify API.spec, test=develop

* fleet api add input check, test=develop

* refactor dygraph,test=develop

* fix failed unittest,test=develop

* polish code,test=develop

* check windows ci error,test=develop
try to fix windows ci error by np.allclose,test=develop

* polish vlog and profiler, test=develop

* try to fix preceding ops order,test=develop

* test transformer in windows ci, test=develop

* use python c-api to speed up tracer.trace,test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, add ut for debug string and gradient_accumulator

* test=develop, add tests for layer/gradient_accumulator/prepared_op

* test=develop, fix complie error for test_prepared_op

* test=develop, add more ut for dygraph

* test=develop, create API.spec for dygraph api change

* test=develop, refoctor name to make it easier to understand

* test=develop, refoctor name to make it easier to understand

* test=develop, fix multi-gpu failed problem , add Tracer tests, change PADDLEENFORCE to PADDLEENFORCE_EQ

* test=develop, fix ut failed on parallel se-resnext

* test=develop, change one more PADDLE_ENFORCE

* add_distributions_of_normal_and_uniform

* paddle/fluid/API.spec

* modify API.spec

* modified paddle/fluid/API.spec, test=develop

* modify paddle/fluid/API.spec, test=develop

* modify paddle/fluid/API.spec, test=develop

* fix some comment, test=develop

* modify API.spec, test=develop

* Add distributions of Categorical and MultivariateNormal test=develop

* fix pylint codestyle test=develop

* fix conflict file test=develop

* edit API.spec test=develop

* improve sample code test=develop

* modify api.spec test=develop

* test=develop add a argument for softshrink python api

* test=develop fix doc format 

test=develop fix doc format

* test=develop fix API.spec

test=develop fix API.spec

* add scatter_nd op, test=document_preview test=develop

* fixed the document, test=document_preview test=develop

* modify the notes, test=document_preview test=develop

* remove the ShareDataWith, test=develop

test=develop, test=document_preview

* add padding in linear_chain_crf op

* modify API.spec

* add linear_chain_crf_op.cc and linear_chain_crf_op.h

* remove useless unit test , test=develop

* modify API.spec, test=develop

* remove some blanks in nn.py , test=develop

* fix some bugs on nn.py and API.spec ,test=develop

* fix nn.py, test=develop

* fix API.spec ,test=develop

* fix bug of CI test in test_linear_chain_crf_op.py

* fix bug of CI test in test_linear_chain_crf_op.py, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* modify nn.py, test=develop

* fix API.spec, test=develop

* fix unittest bug, test=develop

* fixed the code for coverage

* fixed the document,test=document_preview test=develop

* add topk_avg_pooling

* refine api doc and modify api.spec test=develop

Fix mean_iou api misleading example

fix row_conv_op to force it support lodtensor and tensor input simultaneously, test=develop (#19412)

Support Tensor input for row_conv_op

* fix con2d transpose bias by create and init it in build_onee

* fix API spec

* test=develop, invoke ci

* fix bias_attr and act has no effect error on layer norm, conv2dTranpose, billinearTensorProduct, sequece_conv. fix original_mode not used error on GRUunit. fix sample_weight not set error on NCE. Add ut for all thoese layer

* test=develop, change success standard for conv2dTranspose

* test=develop, fix test_layers to invoke some error branch

* test=develop, fix sample code

* test=develop, fix BilinearTensorProduct failed in dygraph mode

* test=develop, fix test_layers segment fault error