Heartbeat for distributed async training.

* add error log

* Update crf_decoding api & example

test=develop

* Update api spec

test=develop

* Fix linear chain crf api

test=develop

* Avoid sharing data pointer with input

test=develop

* Simplify the logic in linear_chain_crf_decoding

* Add unittest for crf_decoding when label & path both are set

test=develop

* Update API spec

test=develop

* Add unittest for layers && correct infer_shape in chunk_eval

test=develop

* fix the error message for reduce_mean and reduce_sum op test=develop

* fix typo test=develop

* fix according review advice test=develop

* fix the test test=develop

* fix test=develop

* fix the constant error message test=develop

* fix typo test=develop

* fix typo test=develop

* fix code style test=develop

* fix comment and bugs test=develop

* fix the bug test=develop

* fix and add unittest test=develop

* fix the typo test=develop

* add support for the fill_constant op test=develop

* add test for ci coverage test=develop

* Add multihead op for ernie opt

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine softmax

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine kernel.

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine cuda kernel

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine cuda version

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine code

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine cmake

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* refector geo sgd & communicator

* Set lod level of sequence_unpad's output to 1 in compile time

Improve elementwise operators performance in same dimensions

* fix compile with anakin bug

* remove useless deps test=develop

- 修复了联编anakin时，遇到的bug.
- 编译test_anakin_activate 不通过
- 编译test_anakin_engine 不通过

1.support asymmetric padding;
    2.support padding algorithm:"SAME" and "VALID";
    3.support channel_last: data_format NHWC and NDHWC;
    4.change doc of python API and c++;

    test=develop, test=document_preview

* refine doc, test=document_fix

* add API.spec,test=develop,test=document_fix

* test=develop, argument shape support tensor and tensor in list

* test=develop,Increasing the coverage of CI tests

* test=develop, modify the document and update API.spec

* test=develop, modify the doc and update API.spec

* test=develop, modify the doc and update API.spec

* test=develop, modify the interface of UniformInitializer

* test=develop, modify the interface of XavierInitializer and MSRAInitializer

* test=develop, modify based on review's comments

* test=develop, modify based on review's comments

*  test=develop, modify based on review's comments

* fix pool2d pool3d:
1. support asymmetric padding;
2. support padding algorithm:"SAME" and "VALID";
3. support channel_last: data_format NHWC and NDHWC;
4. support inferring shape when input with negative dims in compile time;
5. change doc of python API and c++;
6. fix bug in cuda kernel when Attr(adaptive) is true.

test=develop,test=document_preview

* fix 'tensors' to 'Tensors'. test=develop,test=document_preview

* add test for converage ValueError.test=develop,test=document_preview

* resolve conflict in test_pool2d. test=develop

test=develop

* fix pool infershape. test=develop

* fix unittest converage. test=develop

* fix format. test=develop

* make pad and split support fp16
test=develop

* add a base class for the Communicator
* add AsyncCommunicator Impl for async distributed training

Polish English docs of elementwise_add/sub/mul/div

Use AllocateTmpTensor() for creating temporary tensors in warpctc.

test=develop

* Fix conv2d+dequantize squash for residual fusion

test=develop

* Correct int8 input

test=develop

* Add if exclude or include padding in pool2d mkldnn

test=develop

* x.dims == y.dims test=develop

* refine comment

* fix input shape check test=develop

* move PADDLE_ENFORCE test=develop

Add dtype for coalesce_tensor_op

test=develop
test=document_fix

Polish elementwise max min pow document to add more examples

test=develop

* add lod check for sequence op, test=develop

* delete unnecessary check in expend op, test=develop

* add kernel for fill_op, test=develop

* modify PADDLE_ENFORCE to PADDLE_ENFORCE_EQ, test=develop

* add op test for fill_op, test=develop

* REGISTER COP CUDA KERNEL, test=develop

* update test_fill_op.py, test=develop

* change FillConstantOpVarTypeInference to FillOpVarTypeInference, test=develop

* fix op test, test=develop

* add head file, test=develop

* add support tensor and tensorlist for strided_slice OP test=develop

* fix the commnet test=develop

* fix test=develop

* fix the bug test=develop

* delete log test=develop

* fix API.spec test=develop

* fix test=develop

* fix bn

* add support of matmul with multiple head even different width and height

Original matmul with multiple head supports only the mat_a.width == mat_b.height,
in that case, mat_b will be horizontally split. In this patch, we extend the
support when mat_a.width != mat_b.height but mat_a.width/head_number == mat_b.height,
in this case, mab_b will be vertically split.

One example is A is [3, 8], B is [2, 16], head_number is 4. In this
case, A will be split as [3, 2], B will be (vertically) split as
[2, 4]. The final result will be 4 matrix of 4 matrix of [3,4], i.e. [3, 16]

test=develop

* add support of matmul with multiple head even different width and height

Original matmul with multiple head supports only the mat_a.width == mat_b.height,
in that case, mat_b will be horizontally split. In this patch, we extend the
support when mat_a.width != mat_b.height but mat_a.width/head_number == mat_b.height,
in this case, mab_b will be vertically split.

One example is A is [3, 8], B is [2, 16], head_number is 4. In this
case, A will be split as [3, 2], B will be (vertically) split as
[2, 4]. The final result will be 4 matrix of 4 matrix of [3,4], i.e. [3, 16]

test=develop

* refactor the code of matmul with multiple head even different width and height

test=develop