* fix optimizer.state_dict and LRScheduler.state_dict to save/load dygraph,test=develop

* fix optimizer.state_dict and LRScheduler.state_dict to save/load dygraph,test=develop

* Add a judgment that state_dict/set_dict is used incorrectly,test=develop

* fix some doc error,test=develop

* fix current_step_lr for _LearningRateEpochDecay,test=develop

* remove some unsed code to improve coverage,test=develop

* remove some unsed code to improve coverage,test=develop

add new API:LambdaDecay

* add new API: MultiStepDecay, a new learing rate strategy, test=develop

* add new API: MultiStepDecay, a new learing rate strategy,test=develop

* add new API: MultiStepDecay, a new learing rate strategy,test=develop

* add base class of LearningRateEpochDecay, and MultiStepDecay, and StepDecay, test=develop

* fix doc to add coverage,test=develop

添加loss自适应的学习率衰减策略。

* support if logic for Variable in dygraph, test=develop

* fix test_learning_rate_scheduler.py, test=develop

* fix optest, test=develop

* fix error message, test=develop

* fix optest,test=develop

* fix comment, test=develop

* Add parameter(learning_rate) in NoamDecay test=develop

* dygraph mode support linear lr warm up; test=develop

* add unitest for linear warmup; test=develop

* add input type check; test=develop

* fix type check assert error; test=develop

* change type error; test=develop

* deprecate python memory optimize, test=develop

* remove memory_optimize in unittests, test=develop

* add unittests to deprecated interfaces, test=develop

* make it support float/int learning as input.

* Add linear learning warmup method

This warmup lr can be combinated with other learning rate strategies.
For example:
            decayed_lr = fluid.layers.linear_lr_warmup(
                fluid.layers.piecewise_decay(boundaries, lr_steps),
                warmup_steps, start_lr, end_lr)

This reverts commit 3d750f9c.

* wip

* wip

* wip

* wip for test

* add fp16 tests test=develop

* fix cpu build test=develop

* fix test=develop

* fix py3 tests test=develop

* fix lr_scheduler dtype test=develop

* fix test=dvelop

* test fix ci compile test=develop

* fix build and merge test=develop

* fallback momentumop change to general test=develop

* add memopt level

* add opt level for image classification demo

* clean code

* add delete op

* clean code

* test machine translation demo

* clean code

* clean code

* skip fill constant with force cpu

* clean code

* clean code

* refine code

* clean code

* fix bug

* fix mac build error

* override GetExpectedKernelType for cast op

* fix typo

* add cuda unittest

* change learning_rate_decay to learning_rate_scheduler
* optimize code
* change nn.cast to tensor.cast

Currently, our tests run with 2 GPUs, the init time is absurdly long:
about 4s for each process.  Currently, we run each OP test on
different processes. This PR:

1. create cmake function py_test_modules which will generate the
Makefile that runs a list of Python unittest module in a single Python
process.

2. move all "python unittest compatible" (e.g., used the unittest
package, not just a regular python file). from fluid/tests to
fluid/tests/unittests.

3. cmake now will run all OP tests in fluid/tests/unittests in a
single process, except the time-consuming tests, they are separated
into different processes to utilize parallelism. Please make sure to
use the unittest package if you put the python test file in
fluid/tests/unittests

4. remove all exit(0) from fluid/tests/unittests/*.py, exit(0) is used
to disable unittest, we can not do it when running all tests in a
single process since it will terminate the process without running the
other tests. Instead, the test is disabled in
fluid/tests/unittests/CMakeLists.txt. FIXME is added for each disabled
item. Please disable the unittest from
fluid/tests/unittests/CMakeLists.txt, instead of adding exit(0) to the
Python file, for all Python file in fluid/tests/unittests/.

5. add an option WITH_FAST_BUNDLE_TEST. When OFF, will run the unit
tests in separate process so that they can be tested individually.

* init polynomial_decay

* test polynomial_decay

* complete polynomial_decay

* fix conditional block op

* init scalar-switch-case-op

* switch op can compile

* complete forward switch_op

* add GetMatchCaseIndex

* add switch_grad_op

* init switch Python API

* add test_switch

* support set block list in python

* fix scope problem

* complete test

* optimize test

* optimize test

* rm backward part

* clear grad op

* polynomial_decay use switch op

* revert conditional_block_op and reshape_op

* add piecewise_decay and test

* fix piecewise_decay

* try to use condition op for switch

* can work

* clean old code

* revert

* rm switch_op.cc

* optimize code

* add attr is_scalar_condition for condition_block_op

* fix comment

* fix comment

* add export

* add basic interface for learning rate decay
* add exponential_decay
* add natural_exp_decay
* add inverse_time_decay