Optimize amp for multi-gpu to enable FP16 gradients transfer across gpus

* support_dispensable_student_loss, test=develop

* add distillation test, test=develop

* fix distillation test non convergence problem, test=develop

* fix test_distillation fail problem, test=develop

* split teacher checkpoints with student checkpoints, test=develop

* add unittest for graph.merge(), test=develop

Fix float16 optimizer

* delete useless raw variables in Conv2D, test=develop

* adjust the vars number in test_graph_wrapper to pass unittest, test=develop

init new amp, optimize inserting cast op for batchnorm

* test=develop,  communicator merge add => merge average

test=develop

Change backward_guard to optimize_guard to maximize the allreduce overlap

test=develop

* fix bug of getting bool Flags from os.environ, test=develop

* add empty loss_name in CompiledProgram for inplace grad test, test=develop

* python inference api supports numpy and add unit test, fix unit test fail in test_slim_int8_googlenet and test_slim_int8_mobilenet

* update light_nas_strategy: add latency constraint

test=develop

* update light_nas_strategy: update get_model_latency

test=develop

* update light_nas_strategy: add more check

test=develop

* update light_nas test

test=develop

* update light_nas test

    test=develop

* minor update light_nas test

    test=develop

* minor update light_nas test

test=develop

* update light_nas test

test=develop

* update _constrain_func of light_nas_strategy

test=develop

* update _constrain_func of light_nas_strategy

test=develop

* remove unused code

test=develop

* add pool2d quantization support, only for max-pooling.

* add the partial quantization support.

* remove unused inference_transpiler unit-tests

test=develop

* remove InferenceTranspiler usage in quantize_transpiler.py

test=develop

add save cache model api in fleet& add slots shuffle in dataset module & add metric op to calculate ctr related metrics (#18871)

* add ctr related metric layer test=develop

* add save cache and slots shuffle test=develop

* add save cache and slots shuffle test=develop

* fix error

* fix error

* fix style for ci

* fix for comments

* change SlotsShuffle input to std::strinf for generality

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix stylr

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* fix style

* change non-const reference to pointer

* fix style

* fix style

* fix style test=develop

* fix style  test=develop

* add return ins num in ctr metric op

* change dtype to float in metric_op.py

* fix error test=develop

* fix style test=develop

* fix API spec

* fix API spec

* fix API spec test=develop

* add UT test=develop

* update exe.run

test=develop

do some odd jobs, test=develop

* remove ctr reader, all functions are satisfied in dataset

test=develop

test=develop

Socket can't be pickled.
test=develop

* sequnce mask support max length tensor input; test=develop

* add rnn_impl.py; test=develop

* add basic gru lstm unittest; test=develop

* fix api spec; test=develop

* fix sequence_mask op bug;
test=develop
test=document_preview

* change +-*x to elmentwise_op; test=develop

* add mkl flag; test=develop

* fix rnn impl bug; test=develop

* update api spec; test=develop

* fix doc bug; test=develop

* fix lstm bugs; test=develop

test=develop

1. fix the bug that out_put_var in SaveSelectedRows would be empty string
2. use merge_sparse_lookup_table to replace sum op for load_persistables_for_inference
3. fix the bug in _clone_var_in_block_ when the var is SELECTED_ROWS.

* Enable MKL-DNN for slim FP32 vs. INT8 tests

test=develop

* added test for MobileNetV1 with MKL-DNN

test=develop

* QAT doc first version

* follow the comments, test=develop

* update the benchmark, test=develop

* Change to channel-wise, test=develop

* add multi process reader
test=develop

* fix spelling errors; test=develop

* Update API.spec

update md5

* Update API.spec

* change the order of api;test=develop

test=develop

* update CPU_NUM config
test=develop

* added unit test for QAT FP32 & INT8 comparison

test=develop

* enabled other models and updated filenames

test=develop

* added accuracy check and multiple batch handling

test=develop

* removed quantization_mkldnn_pass.py

test=develop

* cleanup

test=develop

* updated model paths

test=develop

* renamed tests without MKL-DNN

test=develop

* fix reusing mkldnn pool2d primitive

test=develop

* add performance measuring

test=develop

* fix accuracy statistics

test=develop

* removed non-mkldnn tests

test=develop

* added conv2d_depthwise->conv2d mkldnn transformation

test=develop

* format update

test=develop

* fixed creating key for pool2d grad

test=develop

* added pass

* Fix the accuracy issue while using float precision to get the scale.

test=develop

* Fix the format issue when 'X' is not nchw.

test=develop

* removed output comparing and changed number of images

test=develop

* cmake and comment fix

test=develop

* updated acc threshold for QAT comparison tests

test=develop

* added OMP_NUM_THREADS setting

test=develop

* enable all QAT INT8 tests

test=develop

* restored upstream version of a file

test=develop

* modified directory names

test=develop