Fix float16 optimizer

* add prune when test_program.clone is executed after optimizer.minimize

* add unittest, test=develop

* add resnet and transformer test case, test=develop

* add regularization for optimizer & program compare function, test=develop

* add lstm unittest, test=develop

* polish code based on review comment, test=develop

* adapt to interface change in framework._prune, test=develop

* update API.spec, test=develop

* add kernel for unsqueeze_op, test=develop

* add kernel for unsqueeze_op, test=develop

* add kernel for unsqueeze_op, test=develop

* delete args on linear_chain_crf_op doc

* delete args on linear_chain_crf_op doc

* delete args on linear_chain_crf_op doc

* add code example

* fix api doc

* fix doc of crf

* fix doc of crf

* add test=develop

* modify API.spec, test=develop

* fleet api add input check, test=develop

* make test_pe_seresnext serial
test=develop

* Increase test_pe_seresnext time limit on MAC
test=develop

* Disable this unit test temporally
test=develop

* Split GPU and CPU
test=develop

* delete useless raw variables in Conv2D, test=develop

* adjust the vars number in test_graph_wrapper to pass unittest, test=develop

init new amp, optimize inserting cast op for batchnorm

* test=develop,  communicator merge add => merge average

* refactor dygraph,test=develop

* fix failed unittest,test=develop

* polish code,test=develop

* check windows ci error,test=develop
try to fix windows ci error by np.allclose,test=develop

* polish vlog and profiler, test=develop

* try to fix preceding ops order,test=develop

* test transformer in windows ci, test=develop

* use python c-api to speed up tracer.trace,test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, add ut for debug string and gradient_accumulator

* test=develop, add tests for layer/gradient_accumulator/prepared_op

* test=develop, fix complie error for test_prepared_op

* test=develop, add more ut for dygraph

* test=develop, create API.spec for dygraph api change

* test=develop, refoctor name to make it easier to understand

* test=develop, refoctor name to make it easier to understand

* test=develop, fix multi-gpu failed problem , add Tracer tests, change PADDLEENFORCE to PADDLEENFORCE_EQ

* test=develop, fix ut failed on parallel se-resnext

* test=develop, change one more PADDLE_ENFORCE

* Fix bug: add feed_vars to the prune function

* Fix data type of variable in edit distance evaluator.
test=develop

* Add unitest for edit_distance python API.
test=develop

* add_distributions_of_normal_and_uniform

* paddle/fluid/API.spec

* modify API.spec

* modified paddle/fluid/API.spec, test=develop

* modify paddle/fluid/API.spec, test=develop

* modify paddle/fluid/API.spec, test=develop

* fix some comment, test=develop

* modify API.spec, test=develop

* Add distributions of Categorical and MultivariateNormal test=develop

* fix pylint codestyle test=develop

* fix conflict file test=develop

* edit API.spec test=develop

* improve sample code test=develop

* modify api.spec test=develop

* enable inplace for affine_channel op, dropout op, test=develop

* remove dropout inplace for ngraph fails, test=develop

* test=develop add a argument for softshrink python api

* test=develop fix doc format 

test=develop fix doc format

* test=develop fix API.spec

test=develop fix API.spec

elementwise broadcast function enhancement

* add kernel for flatten_op, test=develop

* add kernel for flatten_op, test=develop

* fix the license and remove redundant code, test=develop

* enable ngraph throught build_strategy test=develop

* add unittest test=develop

* put use_ngraph unconditional test=develop

* remove paddle_enforce test=develop

* remove paddle_enforce test=develop

* fix copyright test=develop

* limit for ngraph only test=develop

* add scatter_nd op, test=document_preview test=develop

* fixed the document, test=document_preview test=develop

* modify the notes, test=document_preview test=develop

* remove the ShareDataWith, test=develop

* test=develop
Fix the scatter op bug when use the add mode, and support the int64 data type of scatter_op Index(#18804).

* test=develop
Remove the PADDLE_ENFORCE and use PADDLE_ENFORCE_EQ

* test=develop
Remove the fix bug of scatter_add, and just add the support of int64 in scatter_add

* test=develop
Add the test case for scatter op, the test case just for index int64

* add to and detach for Variable in dygraph, test=develop

* add detach for Variable in dygraph, test=develop

* add detach for Variable in dygraph, test=develop

* add detach for Variable in dygraph, test=develop

* add detach for Variable in dygraph, test=develop

* add detach for Variable in dygraph, test=develop

* add exception check, test=develop

test=develop

Disable test_boxps in win32.
Adjust filename to avoid latent multi-thread problem.

* update ngraph ops test=develop

* update unittest test=develop

* increase coverage test=develop

* remove unused PADDLE_ASSERT(_IS_NOT_ERROR)

* replace PADDLE_ASSERT with PADDLE_ASSERT_MSG

test=develop

Change backward_guard to optimize_guard to maximize the allreduce overlap

* fix batchnorm api param: data_layout

* fix batchnorm data_layout param; test=develop

test=develop, test=document_preview

* add padding in linear_chain_crf op

* modify API.spec

* add linear_chain_crf_op.cc and linear_chain_crf_op.h

* remove useless unit test , test=develop

* modify API.spec, test=develop

* remove some blanks in nn.py , test=develop

* fix some bugs on nn.py and API.spec ,test=develop

* fix nn.py, test=develop

* fix API.spec ,test=develop

* fix bug of CI test in test_linear_chain_crf_op.py

* fix bug of CI test in test_linear_chain_crf_op.py, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* remove paddle_enforce, test=develop

* modify nn.py, test=develop

* fix API.spec, test=develop

* fix unittest bug, test=develop

* Support looking up embeddings from BoxPS.
* Add a _pull_box_sparse op, for now this op is not exposed to users.
* Add a BoxHelper class, providing 'BeginPass', 'EndPass', 'FeedPass' functions and so on.
* Add 'BoxPSDataset' in python code.
* Add a compile options WITH_BOX_PS and a MACRO PADDLE_WITH_BOX_PS.
* Add UT.
* More concrete information pls refer to: https://github.com/PaddlePaddle/Paddle/pull/18982

* fixed the code for coverage

* fixed the document,test=document_preview test=develop

- Refactor step 1

- Compilation fix

- Yet another compilation fix

- Even more compilation fix

- Lint fixes

test=develop

- Removed deprectaed PADDLE_ENFORCE occurance

test=develop

- Candidate fix to BN forward

- Lint fixes

test=develop

- Refactoring in data_layout_transform

- compilation fix

- Another comppilation fix

- Step further into darkness

- Yet another compilation fix

- Yet another compilation fix

- missing header

- compilation fix

- Added MKLDNN -> Paddle conversion in fetch op

test=develop

- Compilation fix

test=develop

- Lint

test=develop

- Mul fix

- Fix to MKLDNN MUL op and Elementwise MUL UT

test=develop

- Workaround for diffrent weights with groups representation Paddle vs
  MKL-DNN.

test=develop

- Candidate fix for 5D convolution with groups

- Refactor of fix for conv3d and conv2d in fetch op

test=develop

- Compilation fix

- Still same compilation fix

- Compilation fix

- Compilation fix

- Reverted refactoring of fixes

- Adapted test_conv2d_int8_mkldnn so it exects data in NCHW format
  not NHWC

test=develop

- minor fix in UT

test=develop

- Lint fixes

test=develop

* add topk_avg_pooling

* refine api doc and modify api.spec test=develop