out = w * input
where w is a parameter of size 1

Change-Id: Ife682d62323ceb1a20cbbf6269421b20a862d888

Making '*' support the multiplication between a scalar and LayerOutput

Also changing '+' to support adding between a vector and a scalar.

Change-Id: I7daf35590dc2b2f855a29d9ef43ac57979442e0f

fail in hl_activetype; also fix the hasinputsset in submodel

rebase deconv implementation with develop branch and resolve conflicts with pull#218 commit 45c81a41

This allows user to implement any type of cost by summing over the output of non-cost layers.

Change-Id: Ic55aaabbf0c1299e70b8e48a0effcc91f8f5bd29

* Add elementwise math operations
This allows use to use expressions like: y=log(1+exp(x))
Also added unittests for ActivationFunction
* Enforce keyword arguments for non-positional arguments
* Add LogActivation to doc

* Add benchmark for PaddlePaddle, tensorflow and caffe

* ConvProjection to reduce memory for goolenet

* Add unit test for ConvProjection.
1. unit test in test_LayerGrad.
2. compare the ConvPorjection and CudnnConvLayer, also compare the concat_layer+img_conv_layer and concat_layer_conv_projection.

* Reduce cudnn_conv memory and add benchmark document.
1. Use TmpMatrix as the workspace in cudnn_conv to reduce gpu memory. It reduce lots of memory.
2. Add benchmark document.
3. fix smallnet_mnist_cifar.py in paddle.

* Add job=time and refine cudnn_conv to reduce gpu memroy and speed up

* Refine cudnn_conv and shared biases operation in concat_layer and mixed_layer.

* follow comments

* follow comments

* Use unique_ptr to prevent memory leaks in CudnnConvLayer.

* add maxout layer, including interface and unittest

* follow maxout comments

* auto setting channels

* fix unittest bug in test_RecurrentGradientMachine

* Fix bug in yield dictionary in DataProvider.
* Also make virtualenv work in Paddle.

* support rectangle padding, stride, window and input for PoolProjection

* Follow comments.
1. Remove start
2. refine img_pool_a/b.conf for test_NetworkCompare
3. Split unit test

* Modify the test in img_layers.py

* Fix lots of trainer_config_helpers bug, and complete unittest for `layers.py`

* fix bug in dotmul_operator's api and anotation
* update rnn document
* remove redundant info of projection and operator in layers.py

* remove unnecessary field set in ParameterConfig, Evaluators, etc

* split dotmul_projection and dotmul_operator
* bug fix in outputsize checking for mixed layer

Sequences should be sorted according to the number of subsequences they have.

* modify RecurrentGradientMachine to support hasSubSeq sequence inlinks with the same number of sentence but different number of tokens for each sentence

Change-Id: Ic71f00a4bb346b4fa93e650dfb4b1a0d8d2338b0

Change cos_sim to use CosSimLayer layer when size=1 and rename convex_comb_layer to linear_comb_layer

Also add an unittest for layers.py, currently for a few things. More need to be added later.

* fix CUDNN_VERSION for backward of CudnnBatchNormLayer

* fix cudnn version number for BatchNorm

* the last layer in the stack already has all the suffixes

ISSUE=4586495

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1408 1ad973e4-5ce8-4261-8a94-b56d1f490c56