* Add auto pruning strategy.
1. Fix compressor.
2. Enhence graph executor.
3. Add SAController
4. Add auto pruning strategy.
5. Add unitest for auto pruning strategy.
test=develop

* Init light-nas

* Add light nas.

* Some fix.
test=develop

* Fix sa controller.
test=develop

* Fix unitest of light nas.
test=develop

* Fix setup.py.in and API.spec.
test=develop

* Fix unitest.
1. Fix unitest on windows.
2. Fix package importing in tests directory.

* 1. Remove unused comments.
2. Expose eval_epoch option.
3. Remove unused function in search_agent.
4. Expose max_client_num to yaml file.
5. Move flops constraint to on_epoch_begin function
test=develop

* Fix light nas strategy.
test=develop

* Make controller server stable.
test=develop

* 1. Add try exception to compressor.
2. Remove unitest of light-nas for windows.
test=develop

* Add comments
Enhence controller
test=develop

* Fix comments.
test=develop

* fix cuda/cudnn version detection error, test=develop

* fix again, test=develop

test=develop

test=develop

test=develop

fix bug: the lod_tensor_to_array op will aplly a new var but not release when dong inference (#17856)

test=develop

* save optimizer related vars in dygraph

* test=develop, add optimizer save and load

* test=develop, add optimizer save and load

* test=develop, merge code and add multi-optimizer save and load

* test=develop, fix test_imperative_checkpoint

* test=develop, fix include error

* test=develop, fix include error

* test=develop, renew api spec

* test=develop, refine code

* test=develop, set default value for checkpoint

* test=develop, fix ci error

* test=develop, change API.spec and make api more readable

* test=develop, refine version and time stamp

* test=develop, add example code and refine code

* test=develop, refine doc

* test=develop, change version

* fix the ParallelExecutor on Windows
test=develop
* restrict to use one GPU only under windows

* Enable ngraph layer_norm operator

test=develop

* Disable/Enable cuda, new unit-test test=develop

* Fix use_cudnn test=develop

* Fixed test_layer test, new funciton is added test=develop

* set use_cudnn by default test=develop

* refactor PR 16865

* delete mergetool files

* test=develop

* test=develop

* test=develop

* test=develop

* create dir for int8 model before call SaveOptimModel

* test=develop

* mkldnn int8 only support linux; test=develop

* refine code; test=develop

* remove comment; test=develop

* refine code; test=develop

* fix bug; test=develop

* add exception for mkldnn_post_training_strategy

* reuse int8v2 CAPI dataset; test=develop

* fix accuracy check bug; test=develop

* remove tab

* convert files to unix format

* test=develop

* reduce CI time;test=develop

* reduce CI time and refine code;test=develop

* refine comment; test=develop

* add cmake FLAGS;test=develop

* remove predict_num;test=develop

* use pyreader to read data

* add return_list to PyReader to support return value represented as list

test=develop

* delay infershape test=develop

* fall back subblock to paddle test=develop

* fix edge cases test=develop

* remove output duplicates test=develop

* handle reshape2_grad infershape test=develop

* for debug

* test=develop, memory optimize for dygraph using shared_ptr

* test=develop, fix travis ci showed error

* test=develop, fix bug for recurrent usage of varbase

* test=develop, init varbase when it need to be Add

* test=develop, fix problem of recurrent gradient

* test=develop, add gradient test for recurrent varbase usage

* use python api to reduce python time cost, test=develop

* fix travis ci, test=develop

* fix Py_None error,test=develop

* refine doc for prelu

test=develop

test=develop

test=develop

* update and polish hash op doc

test=develop

* update api spec

test=develop

* expand support tensor attribute; test=develop

* fix bug ; test=develop

* fix uni test bug; test=develop

* fix copy bug; test=develop

* refine expand_times default value; test=develop

* for debug

* test=develop, memory optimize for dygraph using shared_ptr

* test=develop, fix travis ci showed error

* test=develop, fix bug for recurrent usage of varbase

* test=develop, init varbase when it need to be Add

* support some input tensor remain on cpu; test=develop

* fix input = none; test=develop

* fix unfound bug; test=develop

* fix proto None case; test=develop

* fix bug; test=develop

* fix proto null bug; test=develop

* remove conv check; test=develop

* fix test bug; test=develop

* move fill constant; test=develop

* no change in proto; test=develop

* fix bug; test=develop

* change attr detph name; test=develop

* remove remain cpu; test=develop

* fix bug; test=develop

* merge develop; test=develop

* fix one_hot bug; test=develop

* fix bug; test=develop

* fix bug; test=develop

* fix bug; test=develop

* fix python api bug; test=develop

Refactor the code to be OpenMP friendly

test=develop

* revise conv layer cudnn algo choose test=develop

* update for code style test=develop

* update for code style test=develop

test=develop

test=develop

* Enable elementwise_div operator test=develop

* Fix update date test=develop

Note the append_batch_size variable is doing prepend. We should
change the name, but due to backward compatibility, I suggest to
change at v2.0. Not now.

test=develop

* fix the bug of mobilenet-ssd INT8 inference without overloading GetHash
test=develop

* remove the out_grad->format() in TransposeMKLDNNGradOpKernel
test=develop

* add example to get_startup_program()
* fix example to get_startup_program()

* test=develop modify some initializer api

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* test=develop modify API.spec

* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* cache sub_scope, program, var when use_program_cache=True is set

* make fetch_list runable with variables, add more unittest for use_program_cache