* test=develop

* test=develop

test=release/1.5

* Update lamb optimizer (#18333)

* Update lamb optimizer

* Regenerate api spec

test=release/1.5

* Give an experimental warning

test=release/1.5

BackwardStrategy
dygraph.nn
dygraph.checkpoint

*  add cuda_is_available
test=release/1.5

* fix softrelu doc
* update API doc

test=release/1.5

* sequnce mask support max length tensor input; test=develop

* add rnn_impl.py; test=develop

* add basic gru lstm unittest; test=develop

* fix api spec; test=develop

* fix sequence_mask op bug;
test=develop
test=document_preview

* change +-*x to elmentwise_op; test=develop

* add mkl flag; test=develop

* fix rnn impl bug; test=develop

* update api spec; test=develop

* fix doc bug; test=develop

* fix lstm bugs; test=develop

test=release/1.5

* Fix the GetExpectedKernelType of add_position_encoding_op.

* Fix the doc of lstm_unit outputs in nn.py.

* fix default value of fluid.memory_optimize
test=release/1.5

add dygraph api doc for fluidDoc and api spec

* update some op doc, test=release/1.5

* Update backward appending stragety to support double backward and fix some bug. (#18104)

* Update backward.py:
     - If there is no input grad var in all outputs of previous ops, do not append this op into graph.
     - Only apply this stragety when double backward.
* Update some double backward op.
* Update sum_op to judge whether a tensor is empty by numel or IsInitialized().

* add multi process reader
test=release/1.5

* fix save/load in fleet (#17675)

* fix save/load in Fleet
* add UT framework of Fleet (#18058)

* add paddle cloud role maker for customized usage, note this is only for industrial users that have cloud environment pre-configuration (#18121)

add paddle cloud role maker for specific cloud usage. This pr will simplifies user's configuration in distributed training.

* assign role_maker before use (#18137)

Cherry pick retinanet_target_assign_op(#17893), sigmoid_focal_loss_op(#17895) and retinanet_detection_output_op(#17896) for supporting retinanet (#18141)

* test=release/1.5
Fix conflicts in test_layers.py when adding target assign operator for supporting retinanet. Cherry pick #17893

* test=release/1.5
Add sigmoid focal loss operator for supporting retinanet. Cherry pick #17895

* test=release/1.5
Add detection output operator for supporting retinanet. Cherry pick #17896

* test=release/1.5
fix wrong code style in test_layers.py when cherry pick retinanet_target_assign #17893

* test=release/1.5
Fix type error of std::pow in sigmoid_focal_loss. Cherry pick #17895

Add cascade rcnn support.

cherry pick for deform roi pooling

cherry-pick for (https://github.com/PaddlePaddle/Paddle/pull/17402)

Add Pipeline Concurrency Train Mode:
- Cpp: pipeline_trainer & section_worker
- Python: PipelineOptimizer
- Add a new data_feed type: PrivateInstantDataFeed
- Add a test demo of pipeline trainer and the test model is gnn
- Do not support win32 now

* update CPU_NUM config
test=develop

test=release/1.5
cherry-pick from #17952
The scatter op has a calc bug when the indices has same index, the scatter op use overwrite mode to calculate the same index, fix this bug by using the accumulate mode to calculate the same index.At the same time, the gather op has the same bug when the op calc the grad. And we use the lib of open-blas and eigen to optimize the time cost in accumulate mode.

test=release/1.5

* support avx and noavx core
* add catch and give some log
* fix build
* add missing package
* fix pybind name
* fix import error
* conbime noavx core
* add requirements
* fix unkown message
* fix api spec
* refine and clean
* update
* follow comments
* refine scripts

test=develop

* Enable seq_pool op to accept len 0 input

test=develop

* Update sequence_pool's api

test=develop

* Add more unittest cases for seq_pool op

test=develop

* Remove legacy comments

test=develop

* Don't use template in op maker

test=develop

* Add auto pruning strategy.
1. Fix compressor.
2. Enhence graph executor.
3. Add SAController
4. Add auto pruning strategy.
5. Add unitest for auto pruning strategy.
test=develop

* Init light-nas

* Add light nas.

* Some fix.
test=develop

* Fix sa controller.
test=develop

* Fix unitest of light nas.
test=develop

* Fix setup.py.in and API.spec.
test=develop

* Fix unitest.
1. Fix unitest on windows.
2. Fix package importing in tests directory.

* 1. Remove unused comments.
2. Expose eval_epoch option.
3. Remove unused function in search_agent.
4. Expose max_client_num to yaml file.
5. Move flops constraint to on_epoch_begin function
test=develop

* Fix light nas strategy.
test=develop

* Make controller server stable.
test=develop

* 1. Add try exception to compressor.
2. Remove unitest of light-nas for windows.
test=develop

* Add comments
Enhence controller
test=develop

* Fix comments.
test=develop

* save optimizer related vars in dygraph

* test=develop, add optimizer save and load

* test=develop, add optimizer save and load

* test=develop, merge code and add multi-optimizer save and load

* test=develop, fix test_imperative_checkpoint

* test=develop, fix include error

* test=develop, fix include error

* test=develop, renew api spec

* test=develop, refine code

* test=develop, set default value for checkpoint

* test=develop, fix ci error

* test=develop, change API.spec and make api more readable

* test=develop, refine version and time stamp

* test=develop, add example code and refine code

* test=develop, refine doc

* test=develop, change version

* use pyreader to read data

* add return_list to PyReader to support return value represented as list

* refine doc for prelu

* update and polish hash op doc

test=develop

* update api spec

test=develop

Note the append_batch_size variable is doing prepend. We should
change the name, but due to backward compatibility, I suggest to
change at v2.0. Not now.

test=develop

* add example to get_startup_program()
* fix example to get_startup_program()

* test=develop modify some initializer api

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* fix API examples of assign, reverse and array_write
test=develop

* update API.spec
test=develop

* update API examples for array_length, array_read, array_write, assign, hard_sigmoid, hsigmoid, increment, ones, pow, reverse, uniform_random and zeros

* update API.spec for assign, reverse and array_write, etc.(#17287)
* test=develop