out = w * input
where w is a parameter of size 1

Change-Id: Ife682d62323ceb1a20cbbf6269421b20a862d888

Making '*' support the multiplication between a scalar and LayerOutput

Also changing '+' to support adding between a vector and a scalar.

Change-Id: I7daf35590dc2b2f855a29d9ef43ac57979442e0f

And rebase

Change-Id: I7ea234b3aa8fc70675af15d91db08242c43fb5ff

* Add elementwise math operations
This allows use to use expressions like: y=log(1+exp(x))
Also added unittests for ActivationFunction
* Enforce keyword arguments for non-positional arguments
* Add LogActivation to doc

* Add benchmark for PaddlePaddle, tensorflow and caffe

* ConvProjection to reduce memory for goolenet

* Add unit test for ConvProjection.
1. unit test in test_LayerGrad.
2. compare the ConvPorjection and CudnnConvLayer, also compare the concat_layer+img_conv_layer and concat_layer_conv_projection.

* Reduce cudnn_conv memory and add benchmark document.
1. Use TmpMatrix as the workspace in cudnn_conv to reduce gpu memory. It reduce lots of memory.
2. Add benchmark document.
3. fix smallnet_mnist_cifar.py in paddle.

* Add job=time and refine cudnn_conv to reduce gpu memroy and speed up

* Refine cudnn_conv and shared biases operation in concat_layer and mixed_layer.

* follow comments

* follow comments

* Use unique_ptr to prevent memory leaks in CudnnConvLayer.

* add maxout layer, including interface and unittest

* follow maxout comments

* auto setting channels

* fix unittest bug in test_RecurrentGradientMachine

* hierarchical rnn document, add new config example

* update inputs_type of label

* add check for unsupported config

* refine hierarchical document

* refine doc title

* update docs, fix paddle to PaddlePaddle

* follow comments

* add interface and unittest for nce layer

* follow comments

* refine sparse momentum api and unittest
* fix unittests bug

* fix bug in dotmul_operator's api and anotation
* update rnn document
* remove redundant info of projection and operator in layers.py

* min_pool_size would be infinite by default.
  * add unittest for min_pool_size
* Fix bug in can_over_batch_size
  * add unittest for can_over_batch_size
* Add DEFINE_PROVIDER_EX
* Add default value of should_shuffle
  * When training, the default value of should_shuffle is True.
  * When testing, the default value of should_shuffle is False.
  * User a set a provider should_shuffle or not by pass it to `@provider`
  * should_shuffle can handle a list of value, not just boolean
* Add input order mapping by using name
  * Add unittest
* Add check to check input format.
  * Default is close for speed reason.
  * User could stop train when check error, or continue train without
    this train sample.
* use deque instead of vector in generators pool, make erase
  generator faster.
* Add chinese/english documentation
* Make should shuffle = false in unittest
* Add python files to depends.

Change cos_sim to use CosSimLayer layer when size=1 and rename convex_comb_layer to linear_comb_layer

Also add an unittest for layers.py, currently for a few things. More need to be added later.

DataProviderWrapperConverter with DataProviderConverter.

Change-Id: I6ba3ec1daeb3d4aee2556959e87bae7c4c1ad668

ISSUE=4619881 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1502 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4612671

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1501 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4611579

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1473 1ad973e4-5ce8-4261-8a94-b56d1f490c56

Thanks to caoying to check english gramma.

ISSUE=4598269


git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1462 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4603671

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1456 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4604739 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1453 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4589755

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1442 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4597359 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1432 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4592807 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1423 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4584581 

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1413 1ad973e4-5ce8-4261-8a94-b56d1f490c56

ISSUE=4586495

git-svn-id: https://svn.baidu.com/idl/trunk/paddle@1408 1ad973e4-5ce8-4261-8a94-b56d1f490c56