* Adding incremnt op

* Fixing comment about step attribute

* add test_fit_a_line

* Update

* fix persistable bug

* fix elementwise add bug

* set correct attr for bias op in fc layer

* set correct attr for bias op in fc layer

* Update

1. Add init_program to hold initializers
2. bug fix

* add test_fit_a_line

* fix persistable bug

* fix elementwise add bug

* fix type

* add gitignore

* Complete fit_a_line test

* revert code

* Clean up

* Revert "revert code"

This reverts commit eb1aa015.

* Refine

* Fix unit test

* Remove template parameter for Tensor methods

* Also check the type is correct when data()
* Simplize holder_

* Fix accuracy_op

* Register Code

* Implement FC layer with helper

* Update LayerHelper

* Add debug string for Python ProtoBuf

and Rename `Sync` to `Flush`

* Add check of ProtoBuf initialization

* Layer wrapper for FC

* Fix unittest

* Fix CI

* Add code generator

* AttributeChecker Better error log and speicalize bool

Since lots of types can be cast to bool

* Complete mlp, fit_a_line

* Expose get global scope

* Make global scope not thread-safe

1. It is no need to make global scope thread-safe, since it will be
invoked in Python main thread.
2. Do not free the global scope when C++ exit. Let the OS free memories,
otherwise, we need to handle the destroy dependencies.

See
https://google.github.io/styleguide/cppguide.html#Static_and_Global_Variables

* Fix

* Implementation of simple conv_2d layer

* Stash

* Remove private data members in OpRegister

* Fix bugs

* Stash

* Expose FeedFetchList as VarType

* Change ProgramDesc not a global variable

* Polish code style

* Stash

* Correct implement BlockDesc destructor

* Correct implement BlockDesc destructor

* Unify program as parameter name

* Fix bugs

* Add unittest

* Fix unit test error

* Remove unused functions

* Add clone for Python Program

* Working on executor

* Stash

* Add glog as dependencies of ops

* Use VLOG to logging some information is helpful when we debug Paddle

* Expose VarDesc::persistable to Python

* Test executor

* Complete unittest

* Polish code

* Fix merge error

* Follow comment

* Polish Python Code

* Adding Proximal Gradient Descent

* Fixing review comments

* Add design doc of batch_norm_op

* Move batch_norm_op.png to operator/images

* Refine batch_norm_op design doc

* Change ProgramDesc not a global variable

* Polish code style

* Correct implement BlockDesc destructor

* Unify program as parameter name

* initial matmul operator

Similar to np.matmul, but also has transpose_X and transpose_Y flags,
and only supports tensors from rank 1 to 3 inclusive.

For GPU, uses cublas?gemmStridedBatched. For CPU, uses
cblas_?gemm_batch if available via MKL; otherwise a simple serial
implementation that loops over the batch dimension is employed for now.

* init parameter base class

* optimize the Comments of optimizer

* basic implimentation of optimizer

* add test_optimizer

* add no_grad_set to interface

* update optimizer.py

* python code can run

* fix some problem

* add sync_with_cpp to Python Program and Block

* sync vars and ops in block from cpp

* optimize code and add some comment

* add more check for sync

* update optimizer with return value of Backward

* rm unused code

* infer shape when create gradient vairiable

* update test_optimizer

* update test_program.py

* update backward test

* follow comment

They are public now

* Change dataType to data_type

Follow PEP8

* Change name_convention to fit PEP8

* Feed/Fetch op just plain operator, not a OpWithKernel
* Do not register OpInfoMaker since Feed/Fetch will never be
  configured by users
* Feed/Fetch op has empty gradient
* Feed/Fetch op do not hard code `feed_variable`, `fetch_variable` as
  its input and output, make it as a plain Operator input/output

* Adding Hard Sigmoid Activation

* Adding a comment for slope to be only positive

* Fixing grammatical mistake in comment

* Add no_grad_vars for grad_op_maker

* Add unittest

* Fix unittest

* Fix unittest

* Follow comment

* add adam op

moment1_out = beta1 * moment1 + (1 − beta1) * grad
moment2_out = beta2 * moment2 + (1 − beta2) * grad * grad
moment1_hat =  moment1_out / (1 - beta1^t)
moment2_hat =  moment2_out / (1 - beta2^t)
param_out = param - learning_rate * moment1_hat / (sqrt(moment2_hat) +
epsilon)

* fix moment 2

* Adding the Adam optimization operator

* Adding more tests for Adam op