This allows user to implement any type of cost by summing over the output of non-cost layers.

Change-Id: Ic55aaabbf0c1299e70b8e48a0effcc91f8f5bd29

* Add elementwise math operations
This allows use to use expressions like: y=log(1+exp(x))
Also added unittests for ActivationFunction
* Enforce keyword arguments for non-positional arguments
* Add LogActivation to doc

* Add benchmark for PaddlePaddle, tensorflow and caffe

* ConvProjection to reduce memory for goolenet

* Add unit test for ConvProjection.
1. unit test in test_LayerGrad.
2. compare the ConvPorjection and CudnnConvLayer, also compare the concat_layer+img_conv_layer and concat_layer_conv_projection.

* Reduce cudnn_conv memory and add benchmark document.
1. Use TmpMatrix as the workspace in cudnn_conv to reduce gpu memory. It reduce lots of memory.
2. Add benchmark document.
3. fix smallnet_mnist_cifar.py in paddle.

* Add job=time and refine cudnn_conv to reduce gpu memroy and speed up

* Refine cudnn_conv and shared biases operation in concat_layer and mixed_layer.

* follow comments

* follow comments

* Use unique_ptr to prevent memory leaks in CudnnConvLayer.

* fix interface bug of block_expand_layer and add unittest

* auto compute num_channels

* default value of num_channels is None

* adjust input order of block_expand

* add maxout layer, including interface and unittest

* follow maxout comments

* auto setting channels

* fix unittest bug in test_RecurrentGradientMachine

* add interface and unittest for nce layer

* follow comments

* support rectangle padding, stride, window and input for PoolProjection

* Follow comments.
1. Remove start
2. refine img_pool_a/b.conf for test_NetworkCompare
3. Split unit test

* Modify the test in img_layers.py

* add interface and unittest of RecurrentGradientMachine for the function of multiple Subsequence inlinks with unequal token length

* Fix lots of trainer_config_helpers bug, and complete unittest for `layers.py`

* Correctly handling multiple inputs and integer inputs for recurrent_group

* Fix ScatterAgentLayer for generation

* Revert sequence_(nest)_rnn.conf

* Add `device` parameter to ExtraAttr in trainer_config_helpers.
* add unittest for it.

* fix bug in dotmul_operator's api and anotation
* update rnn document
* remove redundant info of projection and operator in layers.py

* split dotmul_projection and dotmul_operator
* bug fix in outputsize checking for mixed layer

Sequences should be sorted according to the number of subsequences they have.

Sequences should be sorted according to the number of subsequences they have.

* Add some comments for CTC layer and fix CTC evalutors, also add interface test

Change cos_sim to use CosSimLayer layer when size=1 and rename convex_comb_layer to linear_comb_layer

Also add an unittest for layers.py, currently for a few things. More need to be added later.

* Fix ParameterAttribute for img_conv_layer, tensor_layer, crf_layer, crf_decoding_layer.

Change-Id: Ic24e4bc1f9d5e03bf35058b49a54c78caf7e97a8

ISSUE=4619881 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1502 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4612671

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1501 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4612433 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1476 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4611081


git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1475 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4611579

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1473 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4599577


git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1440 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4592807 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1423 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4539687

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1422 1ad973e4-5ce8-4261-8a94-b56d1f490c56

Also add some comments for cost layers.
ISSUE=4580653

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1410 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4586495

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1408 1ad973e4-5ce8-4261-8a94-b56d1f490c56