* Update backward appending stragety to support double backward and fix some bug. (#18104)

* Update backward.py:
     - If there is no input grad var in all outputs of previous ops, do not append this op into graph.
     - Only apply this stragety when double backward.
* Update some double backward op.
* Update sum_op to judge whether a tensor is empty by numel or IsInitialized().

* add multi process reader
test=release/1.5

* fix save/load in fleet (#17675)

* fix save/load in Fleet
* add UT framework of Fleet (#18058)

* add paddle cloud role maker for customized usage, note this is only for industrial users that have cloud environment pre-configuration (#18121)

add paddle cloud role maker for specific cloud usage. This pr will simplifies user's configuration in distributed training.

* assign role_maker before use (#18137)

Cherry pick retinanet_target_assign_op(#17893), sigmoid_focal_loss_op(#17895) and retinanet_detection_output_op(#17896) for supporting retinanet (#18141)

* test=release/1.5
Fix conflicts in test_layers.py when adding target assign operator for supporting retinanet. Cherry pick #17893

* test=release/1.5
Add sigmoid focal loss operator for supporting retinanet. Cherry pick #17895

* test=release/1.5
Add detection output operator for supporting retinanet. Cherry pick #17896

* test=release/1.5
fix wrong code style in test_layers.py when cherry pick retinanet_target_assign #17893

* test=release/1.5
Fix type error of std::pow in sigmoid_focal_loss. Cherry pick #17895

Add cascade rcnn support.

cherry pick for deform roi pooling

cherry-pick for (https://github.com/PaddlePaddle/Paddle/pull/17402)

Add Pipeline Concurrency Train Mode:
- Cpp: pipeline_trainer & section_worker
- Python: PipelineOptimizer
- Add a new data_feed type: PrivateInstantDataFeed
- Add a test demo of pipeline trainer and the test model is gnn
- Do not support win32 now

* update CPU_NUM config
test=develop

test=release/1.5
cherry-pick from #17952
The scatter op has a calc bug when the indices has same index, the scatter op use overwrite mode to calculate the same index, fix this bug by using the accumulate mode to calculate the same index.At the same time, the gather op has the same bug when the op calc the grad. And we use the lib of open-blas and eigen to optimize the time cost in accumulate mode.

test=release/1.5

* support avx and noavx core
* add catch and give some log
* fix build
* add missing package
* fix pybind name
* fix import error
* conbime noavx core
* add requirements
* fix unkown message
* fix api spec
* refine and clean
* update
* follow comments
* refine scripts

test=develop

* Enable seq_pool op to accept len 0 input

test=develop

* Update sequence_pool's api

test=develop

* Add more unittest cases for seq_pool op

test=develop

* Remove legacy comments

test=develop

* Don't use template in op maker

test=develop

* Add auto pruning strategy.
1. Fix compressor.
2. Enhence graph executor.
3. Add SAController
4. Add auto pruning strategy.
5. Add unitest for auto pruning strategy.
test=develop

* Init light-nas

* Add light nas.

* Some fix.
test=develop

* Fix sa controller.
test=develop

* Fix unitest of light nas.
test=develop

* Fix setup.py.in and API.spec.
test=develop

* Fix unitest.
1. Fix unitest on windows.
2. Fix package importing in tests directory.

* 1. Remove unused comments.
2. Expose eval_epoch option.
3. Remove unused function in search_agent.
4. Expose max_client_num to yaml file.
5. Move flops constraint to on_epoch_begin function
test=develop

* Fix light nas strategy.
test=develop

* Make controller server stable.
test=develop

* 1. Add try exception to compressor.
2. Remove unitest of light-nas for windows.
test=develop

* Add comments
Enhence controller
test=develop

* Fix comments.
test=develop

* save optimizer related vars in dygraph

* test=develop, add optimizer save and load

* test=develop, add optimizer save and load

* test=develop, merge code and add multi-optimizer save and load

* test=develop, fix test_imperative_checkpoint

* test=develop, fix include error

* test=develop, fix include error

* test=develop, renew api spec

* test=develop, refine code

* test=develop, set default value for checkpoint

* test=develop, fix ci error

* test=develop, change API.spec and make api more readable

* test=develop, refine version and time stamp

* test=develop, add example code and refine code

* test=develop, refine doc

* test=develop, change version

* use pyreader to read data

* add return_list to PyReader to support return value represented as list

* refine doc for prelu

* update and polish hash op doc

test=develop

* update api spec

test=develop

Note the append_batch_size variable is doing prepend. We should
change the name, but due to backward compatibility, I suggest to
change at v2.0. Not now.

test=develop

* add example to get_startup_program()
* fix example to get_startup_program()

* test=develop modify some initializer api

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* fix API examples of assign, reverse and array_write
test=develop

* update API.spec
test=develop

* update API examples for array_length, array_read, array_write, assign, hard_sigmoid, hsigmoid, increment, ones, pow, reverse, uniform_random and zeros

* update API.spec for assign, reverse and array_write, etc.(#17287)
* test=develop

* unit commits, test=develop

* update API.spec, test=develop

* API.spec test=develop

* update

* update test=develop

* update test=develop

* update

* update test=develop

* update test=develop

* update test=develop

* update test=develop

* update test=develop

* test=develop

* update

* update test=develop

* update test=develop

* Enhance fused_elementwise_activation op.
test=develop

* Move the api fused_elementwise_activation to contrib.
test=develop

* Add including files.
test=develop

* Add the support of sigmoid in fused_elementwise_activetion op.

* Update API.spec.
test=develop

* fix:
1. infernce multi card occupy
2. facebox model inference occupy too much
test=develop

* refine python api comments: shuffle, while, scale, sampled_softmax_with_cross_entropy, scatter,
round, sin, sqrt, shape, split, soft_relu, slice, selu, ifelse, switch.
test=develodp

* fix conflict error.
test=develop

* fix doc in transpiler, test=develop

* add gradient clip in minimize; test=develop

* fix bug; test=develop

* fix format; test=develop

* move new grad clip to dygraph/grad_clip.py; test=develop

* fix lr decay and grad clip test; test=develop

* seperate dygraph grad clip; test=develop

* fix grad clip test; develop

* fix api spec bug; test=develop

* add blank line, test=develop,test=document_preview

to fix format problem

* update python APIs

test=document_preview test=develop

* update API.spec

test=document_preview

test=develop

* update merge_selected_rows

* update API.spec

test=document_preview

test=develop

* update API.spec

test=document_preview

test=develop

* fix the comment of less_than

test=develop

test=document_preview

* update API.spec

test=develop

test=document_preview

* update API.spec

test=develop

test=document_preview

* update API.spec

test=develop

test=document_preview

* update API.spec

test=develop

* update API

test=develop

* [DOC] Fix api docs, test=develop, test=document_preview

* [DOC] Fix api annotation: fluid.layers.tensor_array_to_tensor. test=develop, test=document_preview

* test=develop, test=document_preview

update MD5 of tensor_array_to_tensor

* fix the api example for create_global_var, create_parameter, SGDOptimizer, RMSPropOptimizer, MomentumOptimizer, LarsMomentumOptimizer, FtrlOptimizer
test=develop

* add example for adamoptimizer
fix API.spec
test=develop

* test=develop

* test=develop

* Bias correction for exponential moving average

test=develop, test=document_preview

* Fix docs

test=develop, test=document_preview

test=develop

* fix the bug that sub_scope_ may be null in AnalysisPredictor::Run.

* add more directions about io APIs' docs.

* update the API.spec. test=develop test=document_preview