support flexible learning rate adjustment strategy
Created by: jacquesqiao
- learning rate should be a global variable. depends on https://github.com/PaddlePaddle/Paddle/issues/7784
-
define the interface to do learning rate decay. pd-refs tf-refs:
- framework https://github.com/PaddlePaddle/Paddle/pull/7892
- exponential_decay https://github.com/PaddlePaddle/Paddle/pull/7892
- inverse_time_decay #7892
- piecewise_decay #8013
- polynomial_decay #8013
- use condition op to adjust the learning rate during training. #8013 #8184
- add elementwise_pow_op https://github.com/PaddlePaddle/Paddle/pull/7911
-
support syntax like
a+b, a*10. https://github.com/PaddlePaddle/Paddle/pull/7688 - fix floor op https://github.com/PaddlePaddle/Paddle/pull/7926
- what device should learing rate decay happen? should it all happen on CPU? https://github.com/PaddlePaddle/Paddle/pull/8254
- Add switch op #8184