* add new save load interface; test=develop

* add new save interface; test=develop

* add save load interface ;

* fix save load error;

* fix dygraph set dict bug;

* add save load unit test; test=develop

* fix test_imperative_optimizer bug; test=develop

* fix unitest optimizer bug; test=develop

* fix code coverage; test=develop

* fix converage; test=develop

* add document for apis; test=develop

* fix unitest error; test=develop

* fix save load unit test error; test=develop

* fix error message; test=develop

* change set_parameter set_optimizer to save_dygraph; test=develop

* add load_graph check; test=develop

* fix api spec; test=develop

* Update crf_decoding api & example

test=develop

* Update api spec

test=develop

* Fix linear chain crf api

test=develop

* Avoid sharing data pointer with input

test=develop

* Simplify the logic in linear_chain_crf_decoding

* Add unittest for crf_decoding when label & path both are set

test=develop

* Update API spec

test=develop

* Add unittest for layers && correct infer_shape in chunk_eval

test=develop

* opt FC

* opt rest of dygraph.nn

* new param shape check and unittest

* add kwargs for Layer

* add new set_value api

* use property decorator

* update API.spec, test=develop

* use UserList, separate gettersetters, test=develop

* update test_custom_layer_with_kwargs, test=develop

* fix UserList compatibility, test=develop

* fix UserList compatibility, test=develop

* keep FC._w, test=develop

* add unittests, Conv3D bug fix, test=develop

* clean code, test=develop

* fix dygraph guard in unittest, test=develop

* add property setters, remove unused param in tracer, test=develop

* tracer none check, test=develop

* merge, test=develop

* refine, test=develop

* bug fix in  prelu and conv3d_transpose, test=develop

* rm __set__, test=develop

* set tensor value instead of assign op

* fix property setter call, test=develop

* fix api.spec, test=develop

* fix doc sample, test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, Add Variable api and refine dygraph related API

* test=develop, Add Variable api and refine dygraph related API

* test=develop, refine test for new api and error info

* test=develop, refine error info and test_layers

* test=develop, add API.spec

* test=devleop, fix to_string python2 and python3 compat error and refien doc

* test=devleop, add API spec

* test=devleop, update API spec

* test=devleop, update API spec

* test=develop, invoke ci

* test=develop, fix example code

* test=develop, update API spec

* test=develop, add compat test and fix inplace campat dict error

* fix pool infershape. test=develop

* fix unittest converage. test=develop

* fix format. test=develop

* Removing last dims constraints of seq_pad and seq_unpad test=develop

* fix test_layer api code test=develop

* fix sequence_pad_op.cc conflict test=develop

* remove test_analyzer_mm_dnn test=develop

* fix vectorize bug test=develop

* fix vectorize<int> test=develop

* add mse_loss op

* move tree_conv to fluid.contrib.layers

test=develop

* update API.spec for tree_conv

test=develop

* update tree_conv api to increase unit coverage

test=develop

add crop_tensor op. The main difference with crop is :

1. If the argument shape is a list, each element is an integer or a tensor variable with shape: [1]. This way is suitable for the case that the shape may be changed each iteration.

2. If the argument shape is a variable. Its rank must be 1. In crop op, the rank of shape must be the same as x

offsets can be a list, in which each element is an integer or a tensor variavle with shape: [1].

* strided_slice op basic function test=develop

* test=develop rewrite and fix

* fix bug test=develop

* fix for the PADDLE_ENFORCE usage

* add some unit testw

* fix for the aip  test and copright and fix test=develop

* fix API.spec test=develop

* fix API.spec test=develop

* add axis parameter test=develop

* fix for the build error test=develop

* fix python api  test=develop

* fix the build test=develop

* fix build test=develop

* fix API spec test=develop

* test=develop add some comment and single op test

* fix API spece test=develop

* fix test=develop

* fix test=develop

* fix api test=develop

* fix api test=develop

* fix API.spec test=develop

* fix typo test=develop

* fix API.spec test=develop

* fix API typo test=develop

* fix doc and API.spec test=develop

* add deformable conv v1 op, test=develop

* refactor dygraph,test=develop

* fix failed unittest,test=develop

* polish code,test=develop

* check windows ci error,test=develop
try to fix windows ci error by np.allclose,test=develop

* polish vlog and profiler, test=develop

* try to fix preceding ops order,test=develop

* test transformer in windows ci, test=develop

* use python c-api to speed up tracer.trace,test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, add ut for debug string and gradient_accumulator

* test=develop, add tests for layer/gradient_accumulator/prepared_op

* test=develop, fix complie error for test_prepared_op

* test=develop, add more ut for dygraph

* test=develop, create API.spec for dygraph api change

* add transform_data to dygraph

* test=develop, refoctor name to make it easier to understand

* test=develop, refoctor name to make it easier to understand

* add test and change input to const ref for safety

* test=develop, fix multi-gpu failed problem , add Tracer tests, change PADDLEENFORCE to PADDLEENFORCE_EQ

* add ut for data transform

* refine ut for data_transform

* test=develop, fix ut failed on parallel se-resnext

* test=develop, change one more PADDLE_ENFORCE

* add test_tracer on multiple devices

* test=develop, change place to mutable for data transform

* test=develop, add transform data on same place test and remove useless log

* test=develop, Add to do for data layout and and ut for conv2d with no bias

* add unit test for square error cost op

* Fix data type of variable in edit distance evaluator.
test=develop

* Add unitest for edit_distance python API.
test=develop

* test=develop add a argument for softshrink python api

* test=develop fix doc format 

test=develop fix doc format

* test=develop fix API.spec

test=develop fix API.spec

* fix con2d transpose bias by create and init it in build_onee

* fix API spec

* test=develop, invoke ci

* fix bias_attr and act has no effect error on layer norm, conv2dTranpose, billinearTensorProduct, sequece_conv. fix original_mode not used error on GRUunit. fix sample_weight not set error on NCE. Add ut for all thoese layer

* test=develop, change success standard for conv2dTranspose

* test=develop, fix test_layers to invoke some error branch

* test=develop, fix sample code

* test=develop, fix BilinearTensorProduct failed in dygraph mode

* test=develop, fix test_layers segment fault error

* support tensor input with padding for warpctc op

* merge with develop

* test=develop

* modified python API examples test=develop

* nn.py is modified for code coverage test=develop

* update documents info about warpctc op in API.spec test=develop

* add test_warpctc_with_padding in test_layers test=develop

* add warning log for cuda_version back to warpctc_op.cc

* modify API.spec for warpctc op test=develop

* modify API.spec

* update warpctc test to new CompiledProgram API test=develop

* modify code examples for warpctc op test=develop

* modify API.spec for warpctc op test=develop

* modify API.spec for warpctc op test=develop

* instag lod tensor impl

* First PR for instag

* First PR for instag

* Before adding Selection Rows.

* Change name from instag to filter_instag, add upgrade the impl of filter_instag

* Change name from instag to filter_instag, add upgrade the impl of filter_instag

* Fix yapf error in gradient_checker.py to pass Travis-CI

* Fix Filter Instag Grad test=develop

* Fix Filter Instag Grad test=develop

* 1) Fix API.spec, add filter_instag Op. 2) Add Vector Support for CUDA. test=develop

* Impl Loss_weight and empty output handler

* change Loss Weight datatype to Float32, and add Loss Weight as 2nd output

* 1) Support Tensor Input(without LOD) 2) Add Unit test

* Filter By Instag Final test=develop

* Update API.spec for filter_by_instag test=develop

* Update API.spec for filter_by_instag 2 test=develop

* Add Filter By Instag Coverage

* code format of test_layers.py

* code format test_layers.py test=develop

* Make API args more readable test=develop

* Make API args more readable and pass code format test=develop

* Filter By Instag Op, Rename Map to Index Map test=develop

* Filter By Instag Op, code format err in filter_by_instag_op.cc  test=develop

* Filter by instag op: code format of cpp files test=develop

* Filter by instag Op: Api spec modification test=develop

* Filter by instag Op: Api spec doc id modification test=develop

* Filter by instag Op: Api spec and doc preview  test=develop test=document_preview

* Filter By Instag Op, fix doc erro test=document_preview test=develop

* Filter By Instag Op, fix doc err and Api spec test=document_preview test=develop

* Filter By Instag Op, fix Api spec test=document_preview test=develop

* Filter By Instag Op, fix Paddle Encoforce deprecated warning test=document_preview test=develop

* Filter By Instag Op, fix Paddle Encoforce deprecated and code format warning test=document_preview test=develop

* add hard_swish activation op (new op)
test=develop

* remove redundancy files

* modify document content of HardSwish OP

* add API test in test_layers.py

* add dynamic_graph for test_hard_swish

* add eye op,test=document_preview test=develop

* fix the API.spec, test=develop

* fix the document, test=document_preview test=develop

* add unitest for CI coverage, test=develop

* add trilinear interp. test=develop

* fix unittest. test=develop

* add python api and test_layers. test=develop

* refine API.spec. test=develop

* fix format. test=develop

* add python API test. test=develop

* format code. test=develop

* refine code strcuture. test=develop

* fix format

* fix doc. test=develop

* fix converage. test=develop

* fix format. test=develop

* Make lod reset op support for append lod level.

* Fix API.spec
test=develop

* Fix unitest.
test=develop

* Add python api for lod append.
test=develop

* Fix API.spec
test=develop

* Fix format of doc.
test=develop

* Fix unitest.
test=develop

* Fix doc.
test=develop

* test=develop
add detection output for supporting retinanet

* test=develop
add test_layers.py

* test=develop
add API.spec

* test=develop
alter test_retinanet_detection_output.py

* test=develop
alter round 2

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=devlop
alter detection.py

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter detection.py

* test=develop
alter API.spec

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py

* test=develop
alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py

* test=develop
fix grammer error

* test=develop
fix grammer error

* test=develop
fix grammer error

* test=develop
alter python/paddle/fluid/tests/unittests/test_layers.py

* test=develop
alter paddle/fluid/API.spec

* test=develop
add sigmoid_focal_loss for supporting retinanet

* test=develop
add test_layers

* test=develop
add API.spc

* test=develop
alter sigmoid_focal_loss_op.cc

* test=develop
alter detection.py

* test=develop
alter API.spec

* test=develop
alter round 1

* test=develop
alter simooid_focal_loss

* test=develop
alter sigmoid_focal_loss_op.cc

* test=develop
alter test_layers.py

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter sigmoid_focal_loss_op.cu

* test=develop
alter paddle/fluid/operators/detection/sigmoid_focal_loss_op.cc

* test=develop add target assign for retinanet

* test=develop
run ci

* test=developp
add test_layers

* test=develop
add APi.spec

* test=develop
alter round 1

* test=develop
alter rpn_target_assign_op.cc

* test=develop
alter test_rpn_target_assign_op.py

* test=develop
alter rpn_target_assign_op.cc

* test=develop

alter API.spec

* test=develop
alter paddle/fluid/operators/detection/rpn_target_assign_op.cc

* test=develop
alter rpn_target_assign_op.cc

* test=develop
alter python/paddle/fluid/layers/detection.py

* test=develop
alter paddle/fluid/API.spec

* add deformable psroi pooling

* test=develop

* test=develop

* test=develop
modify format

* fix bug

* test=develop run ci

* test=develop
add API.spec

* add test_layers.py

* run ci again

* test=develop
run ci again

* run ci again

* test=develop
run ci again

* test=develop
run ci again

* test=develop
run ci again

* add space between two lines

* test=develop
add space between two lines

* test=develop
add space between lines

* test=develop
modify comment in nn.py

* test=develop
add space between two lines

* test=develop
add space between two lines

* update API.spec

* run ci again

* test=develop
run ci again

* rerun ci

* test=develop
rerun ci

* change input shape

* run ci

* test=develop
run ci

* modify format of nn.py

* test=develop

* test=develop

* test=develop
update API.spec

* test=develop
fix API doc

* modify API comment

* modift API comment

* test=develop
update API.spec

* test=develop
modify comment

* test=develop
modift comment

* test=develop
modift comment

* test=develop
update API.spec

* test=develop
modify comment

* test=develop
add inference in nn.py

* test=develop
update API.spec

* test=develop
resolve confict

* test=develop
update API.spec

* add unfold op
test=develop

* fix divide bug in python3 when calculating output width and height
test=develop

* add name=None in python api, move redundant code into inline function

* try to trigger ci for this code
test=develop

* add api commet; test=develop

* fix fc dtype bug; test=develop

* remove float32 in default parameter; test=develop

* fix exmpale bug; test=develop

* fix build once; test=develop

* fix num_chanels bug; test=develop

* fix install check failed bug; test=develop

* support some input tensor remain on cpu; test=develop

* fix input = none; test=develop

* fix unfound bug; test=develop

* fix proto None case; test=develop

* fix bug; test=develop

* fix proto null bug; test=develop

* remove conv check; test=develop

* fix test bug; test=develop

* move fill constant; test=develop

* no change in proto; test=develop

* fix bug; test=develop

* change attr detph name; test=develop

* remove remain cpu; test=develop

* fix bug; test=develop

* merge develop; test=develop

* fix one_hot bug; test=develop

* fix bug; test=develop

* fix bug; test=develop

* fix bug; test=develop

* fix python api bug; test=develop

* unit commits, test=develop

* update API.spec, test=develop

* add attr axis infershape. test=develop

* add CUDA kernel. test=develop

* fix unittest. test=develop

* fix unittest for soft_label. test=develop

* fix fp16 unittest. test=develop

* remove comment code. test=develop

* refine test for axis. test=develop

* add python api. test=develop

* fix doc. test=develop

* fix fp16 unittest. test=develop

* fix ngraph test. test=develop

* fix ENFORCE for test_imperative_transformer. test=develop

* fit for ngraph test. test=develop

* fix after rebase develop. test=develop

* fix doc. test=develop

* fix API.spec. test=develop

* fix test_layers. test=develop

* fix format. test=develop

Support backward of backward for Relu and add a new gradient checker by comparing theoretical and numerical Jacobian. (#16862)

* Support backward of backward and a new gradient checker
* Rename decorators.py to decorator_helper.py, since Python on Windows CI has decorators package.

1. Add ReluDoubleGradMaker when register relu_grad.
2. Add a new gradient checker by comparing theoretical and numerical Jacobian.  Check double gradients by double_grad_check.

test=develop

test=develop

test=develop

* add pixel_shuffle op

* add pixel_shuffle op, test=develop

* rewrite code, test=develop

* delete useless comment, test=develop

* Refine pixel_shuffle_op and unit testing

* refine code,test=develop

* refine .cu,test=develop

* fix unittest,test=develop

* Fix unit testing
test=develop

* resolve conflict, test=develop

* fix test, test=develop

* fix API, test=develop

* fix test datatype bug,test=develop

* polish comments,test=develop

* add API,test=develop

* test=develop

* Add Pixel_Shuffle OP,test=develop

* support python3,test=develop

* add include memory to travis CI bug,test=develop

test=develop