* add param & grad shape check for sgd op

* add _reshape_inplece interface for dygraph parallel

* refine unittest based paddle/models scripts, test=develop

* add unittest for parallel grad fuse, test=develop

* Commit before merging develop

test=develop

* Backup after working with Huihuang logs

* Commit before deleting Huihuang debug loggings

* Commit before debug

test=develop

* Fix bug commit

test=develop

* Backup of fixing bugs

test=develop

* Clean up code

test=develop

* Fix a bug in sum_op

test=develop

* Add ascending for argsort

* Refine api doc description.

* Refine descending description

* Add int32 logic to speedup when data is small size.

* Remove int32 opt as not support in python

* add_dygraph_execution_context

* add dygraph infershape context and execution context; test=develop

* fix imperative bug; test=develop

* remove inputs outputs interface from execution context,
because it have same function with inputNames;
test=develop

* remove tracer_test ctest; test=develop

* fix split op bug; test=develop

* fix unitests bug; test=develop

* fix distribute test bug; test=develop

* fix ngraph compile bug; test=develop

* fix grad maker bug; test=develop

* fix load op bugs; test=develop

* fix operator.cc construct bug; test=develop

* remove useless name find in operator; test=develop

* add tracer_test; test=develop

* fix concat, split bug; test=develop

* remove tracer_test unitest; test=develop

* fix attribute check bug; test=develop

* add test code to fix converage; test=develop

* remove useless code, change check backward input in engin; test=develop

* unlock var type infer shape;test=develop

* add ShareAllLoD api; test=develop

* add dygraph infershape context unitest; test=develop

* remove increase and decrease lod in dygraph; test=develop

* addd override; test=develop

* fix increase descrease lod; test=develop

* fix paddle_enforce; test=develop

* disable lod op dygraph check; test=develop

* fix paddle enforce error; test=develop

* add comment for op_registry and OperatorBase; test=develop

* optimize the comment of op_registry; test=develop

* fix format of comment; test=develop

* fix format of comment; test=develop

* optimize the format of comment; test=develop

* optimize the format of the comment; test=develop

* optimize comment of op_registry; test=develop

remove nccl related code in windows

* add Adam beta1/beta2 support Variable. test=develop

* use system allocator in unittests, test=develop

* fix op bugs, test=develop

* fix tensor copy bug when src and dst are the same, test=develop

* batch_norm momentum support variable. test=develop

* fix format. test=develop

* add batch_norm momentum variable example. test=develop

* move MomentumTensor to training branch. test=develop

* split example. test=develop

* fix doc. test=develop

* fix PADDLE_ENFORCE ci. test=develop

* fix format. test=develop

* support data_norm_op run in CUDA
* add two parameters sync_stats & summary_decay_rate
* add UT

* add numpy bridge

* fix template compile

* add unittest, add default
test=develop

* fix unittest
test=develop

* fix unittest
test=develop

* zero_copy=True for to_variable,
test=develop

* bug fix
test=develop

* disable deprecated NumPy API
test=develop

* use better design of NumpyAllocator
test=develop

* fix Py_None check
test=develop

* reset c++ tracer when jump out dygraph guard
test=develop

* refine PADDLE_ENFORCE_xx format
test=develop

* bug fix of tracer switch
test=develop

* update decref
test=develop

* Implement Int8 FC

* Integrate FC into INT8v2

test=develop

* int8 FC: transpose weights before computing scales

test=develop

* Add support for activation_type string in FC

test=develop

* Disable MKL-DNN's FC in VGG16 and 19

test=develop

* Disable FC quantization when mkldnn FC is disabled

test=develop

* Solve PADDLE_ENFORCES in FC int8

* Fix Paddle enforces and remove const cast

test=develop

* Fix style changes

test=develop

* Fix quantizer_tester test and add fc quantization

test=develop

* Fix FC test fail on CUDA

* Remove unnecessary log from quantize placement pass

test=develop

* Add Thread ID to FC hash key

test=develop

* Add comments to MKL-DNN FC Kernel

test=develop

* Refactor quantizer

test=develop

* Fix linter issues

test=develop

* Fix crash in slim googlenet

test=develop

* Fix PADDLE_ENFORCE messages

test=develop

* add the framework support for distfc and ut, test=develop
* fix the implementation of shard_index_op, test=develop

* change download log formate; test=develop

* add unittest for data download; test=develop

* remove cache before download; test=develop

* Add fc padding to solve mkl performance
test=develop

* fix gpu pass and error information
test=develop

* fix fc_fuse_pass_test
test=develop

* fix error information
test=develop

* fix error information
test=develop

* fix name and add fc op padding test
test=develop

* fix attributes
test=develop

* optimize fc padding
test=develop

* fix test
test=develop

* Refactor MKL-DNN ElementwiseMul

remove manual fallback, remove format attrs
test=develop

* Refine PADDLE_ENFORCEs in eltwise_mul_op.h

test=develop

* Make ElementwiseMulOp inherit from ElementwiseOp

* Change type of simd_width to int

test=develop

* Remove Constructor extensions in ElementwiseOp and ElementwiseMulOp

test=develop

* Restore attributes

test=develop

* Fix test coverage for mkldnn eltwise mul

test=develop

* Conform to new is_run_common_broadcast API

test=develop

* Add UT for AreDimsAndFormatCorrect

test=develop

* Improve argsort performance.

- Give 200000 data to compute argsort on v100,
can speed up ~190x
before opt cost: 0.53s
after opt cost:0.0027s

- Add fp16 support

* Refine error message
* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* add global value getter setter, test=develop

* fix error messages, test=develop

* fix fetch handler problem and refactor
when a user define FetchHandler class, he or she should initialize a handler
with variable dict. the key of a variable dict is a user defined name,
the value of a variable dict is a Varaible generated from python API.

For each fetching, a user should implement handler function in which
fetched_result_dict will be available and the user can access the fetched value
with user defined keys.

* adapt test_collective_base.py for only two GPU cards available.
test=develop

* fix bug of issue #21259
test=develop

* add int8 kernel to lookup_table op and add dequantize op test=develop

* change paddle_enforce to paddle_enforce_eq test=develop

* change copyright and change some not suitable code test=develop

* remove debug log test=develop

* replace GetInputType with IndicateVarDataType test=develop

* fix EmptyGradMaker test=develop

* fix diff between cpu and gpu test=develop

* use memcopy when int8_t test=develop

Previously, CVM OP was only able to run in CPU. This PR implements its GPU kernel.
What's more, we improve the UTs about CVM OP.

* polish code details, test=develop

* futher polish hint msg, test=develop

* fix fs_client_param bug， user can set this config through fleet_desc_file or fleet config
* test=develop

* open dygraph op test, test=develop

* modify to_variable, test=develop

* modify input and output for dygraph, test=develop

* modify input and output for dygraph(fix bug), test=develop

* fix input processing of dygraph op test, test=develop

* fix bug, test=develop

* fix op test, test=develop

* fix forward bug for dygraph, test=develop

* fix mkldnn op test for forward, test=develop

* update nn.py for dygraph, test=develop

* fix crop_tensor_op, test=develop

* fix elementwise_mul_op, test=develop

* fix fill_op, test=develop

* fix some mkldnn op, test=develop

* open backward op test for dygraph, test=develop

* delete log, test=develop

* close backward op test for dygraph, test=develop

* fix bug for edit_distance_op and test_lstm_cudnn_op, test=develop

* fix optest backward bug for dygraph, test=develop

* fix optest backward bug for dygraph, test=develop

* close backward op test for dygraph, test=develop

* close backward op test for dygraph, test=develop

* open dygraph op test, test=develop

* fix op test for dygraph, fix GradOpDescMaker, test=develop

* fix bug for linear_chain_crf_op.h, test=develop

* remove log, test=develop

* remove log, test=develop

* remove log for op_test.py, test=develop

* remove log for op_test.py, test=develop

* fix bug for var_conv_2d_op, change PADDLE_ENFORCE, test=develop

* fix PADDLE_ENFORCE_EQ for hierarchical_sigmoid_op.cc, test=develop

* fix bug for test_increment_ngraph_op.py, test=develop

* fix lod for op test in dygraph, test=develop

* refactor op_test.py to reduce redundant code, test=develop

* fix lod optest, modify InputVar/OutputVar to HasInput/HasOutput, test=develop

* remove debug log, test=develop

* remove redundant code in base.py, test=develop

* fix some error in optest, test=develop

* fix ClearNoNeedBufferInputs function's bug for LoDTensor, test=develop

* refactor op_test.py, test=develop

* remove redundant writing, test=develop

* fix error(get tensor of the grad variable), test=develop

* fix test_concat_mkldnn test_conv2d_mkldnn, test=develop

* fix optest.py for get tensor of LoDTensor, test=develop

* fix optest.py for get tensor of LoDTensor, test=develop

* fix optest.py for get tensor of LoDTensor, test=develop

* fix some redundant code, test=develop

* reslove conflict and rewrite paddle error message, test=develop

* add input type and input data type check for Print_op test=develop

* add control flow API: case. test=develop

* delete 'raise TypeError' in _error_message() and return a string. test=develop

* polish API document. test=develop

* fix auc drop first commit test=develop

* update datanorm op

* update datanorm with enforce test=develop

* update test=develop

* update format test=develop

* update format

* update format test=develop

* add unit test test=develop

* update unit test test=develop

* update format test=develop

* update format test=develop

* update API description test=develop

* update API description test=develop

* update format test=develop

* fix codes as comments test=develop

* fix description as comments test=develop

* fix description as comments test=develop

* update codes.. test=develop

* fix warnings oof gcc 8 compilation, test=develop

* fix boost::bad_get, test=develop

* refine PADDLE_ENFORCE, test=develop

* add API switch_case. test=develop

add Nest

* modify code according to reviews:
1.Attr(branch_index) support 'uint8' and 'int64' besides 'int32'.
2.remove useless code.
test=develop

* replace fluid.layers.data with fluid.data and polish API document. test=develop