* "add c++ side kernel selection"

* "add multiple kernel op test"

* "kernel selection only support cudnn"

* "better formatter"

* "small fix with UseCPU"

* "depends on change interface Get(Place, Library)"

* "fix CI"

* "fix python cudnn test"

* "leave the register cudnn op to another PR"

* "fix CI"

* "use all kernel by default"

* "fix CI"

* init kernel hint

* fix typo

* rm unused code

* add include in op_kernel.h

* restore op_kernel since it will be moved to op_kernel_type

* change force_cpu to use_cpu

* fix compilation

It is useful to reorder RNN memory block.

* Support Clip in param_attr

* Fix the order of clip & regular

Regular is not need to be clipped

* "add init seed"

* "fix compile error"

* "add program level seed setting"

* "fixed based on comments"

* add crf_decoding layer

* fix some typo

* fix test_crf_decoding_op

* Unify fluid submodules to fluid module

Change books just use `import fluid`, not submodules

* Remove g_main_program/g_startup_program

Use default_main_program/default_startup_program instead

* Typo

* Add API for switch default program

* Two functions: switch_main_program/switch_startup_program
* A guard: program_guard. Users can use the `with` statement change
  default programs
* Change unittests in `test_layers`

* Fix CI

* Fix CI

* Fix CI

* Unify fluid submodules to fluid module

Change books just use `import fluid`, not submodules

* Remove g_main_program/g_startup_program

Use default_main_program/default_startup_program instead

* Typo

* Fix CI

* Restore the param infos in Program.clone()

The Program.clone only clone the variables and ops
in the program into a new program. However, the
information of Parameter is not clone.

So we need restore the information of Parameters.

Fix #5871

* Follow comments

* Fix CI

* Fix CI

* Fix CI

Change books just use `import fluid`, not submodules

* is_training to is_test in dropout op

* handle dropout and batch_norm operator when prune pdesc in testing mode

* handle dropout and batch_norm operator when prune pdesc in testing mode

* add get_inference_program method

* fix dropout op

* fix ci

* test data after each batch training

* refine code

* refine test_book3

* fix ci

* follow comments

* Change all `data_type` in Python to `dtype`

* Change `date_type` in C++ to `dtype`

* Refine

* Improve the initializer Interface for fc, sequence_conv and conv2d layers
* Fix some typos in python code
* Fix CI

* first commit

* Python API for while op

* Python Unittest for simple while_op forward

* fix out to be list

* Fix UT

* VarType

* Fix several bugs

* Fix bug

* Fix bug

* Fix Bug

* Fix bug

* Fix unittest

* Remove debug log

* Add comments

* add PADDLE_ENFORCE

* while_grad_op first commit

* Add `BlockDescBind::FindRecursiveOrCreateVar()` and fix bugs

* not sure how to setdim of while outputs

* push for test

* add executor vlog

* fix bug of while_op cond

* Several enhancement for code

1. Backward always infer shape & infer var type. Since there are RENAME
variables will be created when creating backward operator, but their
shape & var types are not inferenced.
2. Never use SomePtr-> directly, since every pointer could be nullptr if
it is a function return value. Add `detail::Ref` to cast pointer to
reference safely.
3. Enhance error message for backward.
4. Infer data type of variable in `sum` and `tensor_write`

* Fix bugs of while_op gradient

* Fix several bugs of while_op grad

* fix fill zeros like

* fix 3 >= 3

* fix place holder shouldn't be null

* fail on sum op

* Fix SumOp of TensorList

* clean up

* pass while test

* fix test_array_write_read

* pass sum op

* Support int/int64 for fill_constant_batch_size_like

* Fix compile

- Removed all main_program and startup_program in the demo.
- Using paddle.default_main_program() hides the implementation detail (e.g., using g_main_program) from the user, we can change the implementation in the future much easier.

* init commit

* change some dir name

* Conditional Block Forward

* Assign Operator.

Out=X, when type in [LoDTensor/SelectedRows/LoDTensorArray]

* Stash

* Add Scope::Rename

it is useful in gradient phase of an operator with block

* ConditionalBlock Grad Done

* Add comments

* yapf format code

* first commit

* Python API for while op

* Python Unittest for simple while_op forward

* fix out to be list

* Fix UT

* VarType

* Fix several bugs

* Fix bug

* Fix bug

* Fix Bug

* Fix bug

* Fix unittest

* Remove debug log

* Add comments

* add PADDLE_ENFORCE

* while_grad_op first commit

* Add `BlockDescBind::FindRecursiveOrCreateVar()` and fix bugs

* refine code

* fix unittest bug

* Use stable_sort in lod_rank_table

It is easy to debug and test when use `stable_sort`and the time
complexity is not changed.

* Add LoDTensorArray

* Stash

* Better debug message for IsInitialized

* Stash

* Better debug message for IsInitialized

* Complete array read/write op unittests

* Add LoDRankTable

LoD Rank Table stores the `level` of `lod` which is ordered by sequence
length in descending order. It is useful when implement dynamic RNN and
is shared by dynamic RNN memory, dynamic RNN slice input and dynamic
RNN slice output operators.

* Add InferVarType

* Init commit

* Make executor use ProgramDescBind

* Change Attribute from BlockDesc to BlockDescBind

* Since we will get the program desc in RNN, just BlockDesc is not
  enough.

* Add DeviceContext to Executor API

* Rewrite RNN

* Pass Python

* AddBiasOp does not care num_flatten_dims

* Stash

* Fix MacOS Compile

* Pass RNN forward

* add python test

* refactor test

* Make compile pass

* add gradopmaker

* First draft done

* Polish code

* add grad op maker and grad infershape

* Polish code

* Fix backward.cc bug

* Fix infershape

* Rename function

* add backward test

* simplify recurrent test

* Update

* Pass unittest

* Add comments & refine test

* Add comments

* refactor test

* Complete Unittest

* fix StepScopes enforce

* Remove unused unittest

* no type error

* Update

* Make RNN Pass unittest

* optimizer use init_program

* create persistable variable

* add create_persistable_var to block

* optimizer use create_persistable_var

* fix prefix

* move create_global_persistable_var from Block to LayerHelper

* Polish Optimizer initialization code.

* Using the LayerHelper to create initialize operator and variables

* add_accumulator should use an independent data type

* default use param data type for accumulator