* add data parallel batch

* add quant_dequant_pass, test=develop

* Add quant_dequant before some ops, such as the elementwise_add op. This is required by TensorRT. test=develop

* Add LAMB optimizer

* Expose LAMB Optimizer's APIs

test=develop, test=document_preview

* Cleanup code & doc

test=develop, test=document_preview

* Update lamb optimizer's formula

test=develop

* init auto loss scaling

test=develop

* change API.spec

* change ifelse to switch and use reduce_sum to optimize checking isfinite

test=develop

* Remove redundant code

test=develop

* resolve #17184

Fix the KL algorithm bug when calculated the size of tensor.

* Add the missing test=develop.

test=develop

test_distillation_strategy always failed on a machine with 4 gpus only, disable temporarily and need to figure out the root cause and add it back later

Update the folder name generation mechanism for saving the quantized model and weights.
The folder name would be unique by adding the timestamp postfix.

test=develop

* Init mixed precision training interface

* Add fp16 test script

test=develop

* All initializers support float16

test=develop

* Code cleanup & add more code annotations

test=develop

* Update API spec

test=develop

* Add usage example in doc

test=develop

* Split test_calibration test to two tests

test=develop

* Modify CMakeLists file
test=develop

test=develop

test=develop

test=develop

* Add DecoupledWeightDecay

test=develop

* Add a tool to summary model's PARAMS, FLOPs in paddle/fluid/contrib.

test=develop

test=develop

* fix cdn issue, test=develop

* fix memory optimize bugs, test=develop

* fix memory optimize bugs, test=develop

* remove add/sub_2 op, test=develop

* disable memory_optimize by default, test=develop

* disable inplace activation in python, test=develop

* fix unittests, test=develop

* fix unittests, test=develop

* bug-fix, test=develop

test=develop

* Update Readme with new accuracy and performance data measured on 6271
test=develop

* Modify tables and example command
test=develop

* Add fsp operator.
1 Add unitest.
2. Add python API.
3. Add layer test.

* Add quantization strategy.
1. Add API.
2. Add unitest.

* Add distillatoin strategy.

* Add unitest config file for quantization

* Fix Copyright
test=develop

* Fix setup.py

* Fix document of layers.py.
test=develop

* Fix unitest in python3.
test=develop

* Fix documents.
test=develop

* 1. refine fsp op by batched gemm
2. remove unused import
test=develop

* Fix test_dist_se_resnext.
1. disable test distillation.
2. reset framework.py
test=develop

* Enable unitest of distillation after fixing Block._clone_variable
test=develop

* Fix cdn issue.
test=develop

* fix cdn issue, test=develop

* fix cdn issue, test=develop

test=develop

* First pr of paddle slim.
1. Add framework of paddle slim
2. Add filter pruning strategy
test=develop

* Rename unitest to tests.
test=develop

* Add prettytable into requirements.
test=develop

* Change in_nodes and out_nodes to odered dict.
test=develop

* Remove distillation.
test=develop

* Fix API.spec
test=develop

* Fix unitest.
test=develop

* Fix unitest in windows.
test=develop

* Fix unitest in windows.
test=develop

* Fix unitest.
test=develop

* Hide some functions.
test=develop

* Fix python import in python3.5
test=develop

* Fix compress pass.
test=develop

* Fix unitest of test_dist_ctr.
test=develop

* Enhence flops.

* use os.path.join

* Fix pickle for python3
Fix log and comments.
test=develop

* 1. Remove feed_reader in compress pass
2. Fix cache reader
3. Rename CompressPass to Compressor
4. Add comments for distiller optimizer
5. Remove unused pruner currently
6. Add some comments.
7. Change API.spec
test=develop

* Fix pruning in python3.
test=develop

* Fix unitest in python3.
test=develop

* Fix format in python3.
test=develop

test=develop

* update load persistables for increment, test=develop

* update load persistables for increment, test=develop

* update API Spec, test=develop

* update API Spec, test=develop

* add doc, test=develop

* add doc, test=develop

* Update lookup_table_utils.py

* Update API.spec

* Update lookup_table_utils.py

test=develop

* Update API.spec

test=develop

* fix api spec

* Update lookup_table_utils.py

test=develop

* Add moving average absmax op in quantilize-aware training.