Follow comments

Used for shrink memories state in DyRNN. The height of state could
be shrinked after running a step block.

* Use stable_sort in lod_rank_table

It is easy to debug and test when use `stable_sort`and the time
complexity is not changed.

* Add LoDTensorArray

* Stash

* Better debug message for IsInitialized

* Stash

* Better debug message for IsInitialized

* Complete array read/write op unittests

* Add unittest, Gradient of array read/write

* Follow comments

* Use stable_sort in lod_rank_table

It is easy to debug and test when use `stable_sort`and the time
complexity is not changed.

* Add LoDTensorArray

* Stash

* Better debug message for IsInitialized

* Stash

* Better debug message for IsInitialized

* Complete array read/write op unittests

* add acc layer
* memory log level change from 3 to 10
* use gaussian random to init conv parameters
* use initializer
* fix import
* batch_norm use helper to create persistable var
* refine code
* train only 2 batches for test
* use g_program and g_init_program
* use XavierInitializer to init fc parameter

* Add LoDRankTable

LoD Rank Table stores the `level` of `lod` which is ordered by sequence
length in descending order. It is useful when implement dynamic RNN and
is shared by dynamic RNN memory, dynamic RNN slice input and dynamic
RNN slice output operators.

* Add InferVarType

* Fix bug in lookup_table_op & layers

* Missing Act in layers

* Should += in CPU

* Remove check in python

* Fix bug in sequence_conv_pool()

* Fix a bug in test_recommender_system.py

* Just skip test_evaluator

* Init commit

* Make executor use ProgramDescBind

* Change Attribute from BlockDesc to BlockDescBind

* Since we will get the program desc in RNN, just BlockDesc is not
  enough.

* Add DeviceContext to Executor API

* Rewrite RNN

* Pass Python

* AddBiasOp does not care num_flatten_dims

* Stash

* Fix MacOS Compile

* Pass RNN forward

* add python test

* refactor test

* Make compile pass

* add gradopmaker

* First draft done

* Polish code

* add grad op maker and grad infershape

* Polish code

* Fix backward.cc bug

* Fix infershape

* Rename function

* add backward test

* simplify recurrent test

* Update

* Pass unittest

* Add comments & refine test

* Add comments

* refactor test

* Complete Unittest

* fix StepScopes enforce

* Remove unused unittest

* no type error

* Update

* Make RNN Pass unittest

* "add sequence conv layer"

* "add book recommender_system testing"

* "add training loop"

* "add sequence layer"

* "add recommender system training data"

* "fix conv2d layer bug"

* add sequence_conv_pool

* "fix input is Null"

* add networks

* "fix based comment"

* "add sum op layer"

* "merge layers"

* Update layers.py

* "fix input is NULL bug"

* "debug embedding table"

* "modify layers.py"

* "fix pool interface"

* "add export type to layers"

* "fix based on comment"

* "need lod info support in all operator"

* "remove accuracy layer"

* "tuning learning rate"

* "add sparse test"

* "add gpu test"

* Update test_recommender_system.py

* Fix bug

* update

* "replace enum with string"

* "fix layers"

* "modify layers.py"

* "fix pool interface"

* "add export type to layers"

* "fix based on comment"

* refine evaluator op types

* update

* follow comments

* update

* fix v2 mnist case

* fix v2 mnist case

* update

* update

* "modify layers.py"

* "fix pool interface"

* "add export type to layers"

* "fix based on comment"

* refine evaluator op types

* update

* follow comments

* update

* fix v2 mnist case

* fix v2 mnist case

* update

* update

* Complete accuray layer

* Fix error

* Fix error

* Add 'accuracy' to __all__

* update

* Fix Type error

* Fix error

* Refine unit tests

* Fix an unit test error

* reimplement pow operator

* add pow_grad operator

* fix code style

* fix build error

* fix op_test bug

* revert pow operator

* add FIXME comment

* "add sequence conv layer"

* "add sequence layer"

* add networks

* "fix based comment"

* Update layers.py

* add resnet

* optimize code

* add batch_norm_layer

* add img_conv_group layer and test

* add check to Tensor.type()

* forward can run

* with backward

* change label data time from int32 to int64

* refine code

* follow comment

* Cast Operator

Cast input variable to other data type

* Fix compile error

* Add cast op

* Follow comments

* add sparse support for sum op

* typo fix

* fix gpu build error

* fix unittest error

* typo fix

* infer var type and shape in op_test

* follow comments

* fix build error

* bypass some unittests depend on NetOp

* support sparse output for lookup table grad op

* refine codes

* fix gpu build error

* fix lookup table grad gpu kernel

* fix ci

* fix ci

* fix ci

* fix bug in lookup_table_grad op

* fix bug in test_word2vec

* register double kernel for some operators

* set is_sparse=True in test_word2vec

* fix lookup table grad op CUDA kernel bug

* disable test_modified_huber_loss_op temporarily

* disable test_lstm_unit_op temporarily

* init

* ensure ids in lookup table op must be a column vector

* add book4 configuration in test_layers

* debug test_book4

* add test_word2vec

* follow comments

* follow comments

* add test_fit_a_line

* Update

* fix persistable bug

* fix elementwise add bug

* set correct attr for bias op in fc layer

* set correct attr for bias op in fc layer

* Update

1. Add init_program to hold initializers
2. bug fix

* add test_fit_a_line

* fix persistable bug

* fix elementwise add bug

* fix type

* add gitignore

* Complete fit_a_line test

* revert code

* Clean up

* Revert "revert code"

This reverts commit eb1aa015.

* Refine

* Fix unit test

* Init

* unify layer names

* Update

* Add pool2d layer

* Test recognize_digits_conv

* Clean up

* Implement FC layer with helper

* Update LayerHelper

* Add debug string for Python ProtoBuf

and Rename `Sync` to `Flush`

* Add check of ProtoBuf initialization

* Layer wrapper for FC

* Fix unittest

* Fix CI

* Add code generator

* AttributeChecker Better error log and speicalize bool

Since lots of types can be cast to bool

* Complete mlp, fit_a_line

* Expose get global scope

* Make global scope not thread-safe

1. It is no need to make global scope thread-safe, since it will be
invoked in Python main thread.
2. Do not free the global scope when C++ exit. Let the OS free memories,
otherwise, we need to handle the destroy dependencies.

See
https://google.github.io/styleguide/cppguide.html#Static_and_Global_Variables

* Fix

* Implementation of simple conv_2d layer

* Stash

* Remove private data members in OpRegister

* Fix bugs

* Stash

* Expose FeedFetchList as VarType

* Change ProgramDesc not a global variable

* Polish code style

* Stash

* Correct implement BlockDesc destructor

* Correct implement BlockDesc destructor

* Unify program as parameter name

* Fix bugs

* Add unittest

* Fix unit test error

* Remove unused functions

* Add clone for Python Program

* Working on executor

* Stash

* Add glog as dependencies of ops

* Use VLOG to logging some information is helpful when we debug Paddle

* Expose VarDesc::persistable to Python

* Test executor

* Complete unittest

* Polish code

* Fix merge error

* Follow comment

* Polish Python Code

* Implement FC layer with helper

* Update LayerHelper

* Add debug string for Python ProtoBuf

and Rename `Sync` to `Flush`

* Add check of ProtoBuf initialization

* Layer wrapper for FC

* Fix unittest

* Fix CI

* Add code generator

* AttributeChecker Better error log and speicalize bool

Since lots of types can be cast to bool

* Complete mlp, fit_a_line

* Implementation of simple conv_2d layer

* Fix bugs

* Remove debug code

* Implement FC layer with helper

* Update LayerHelper

* Add debug string for Python ProtoBuf

and Rename `Sync` to `Flush`

* Add check of ProtoBuf initialization

* Layer wrapper for FC

* Fix unittest

* Fix CI

* Add code generator

* AttributeChecker Better error log and speicalize bool

Since lots of types can be cast to bool

* Complete mlp, fit_a_line