Created by: jacquesqiao

Background

@seiriosPlus find a bug in the new Trainer high-level API when writing a model with learning rate decay. The optimizer is created in a different program than the trainer use.

Reason and solution

https://github.com/PaddlePaddle/Paddle/blob/9f0dcfd40c048fef6a57fbf4816a38074da1b1fc/python/paddle/fluid/trainer.py#L88-L103

In the design of Trainer API, train_program is a function call, it will be called inside Trainer under a with scope https://github.com/PaddlePaddle/Paddle/blob/9f0dcfd40c048fef6a57fbf4816a38074da1b1fc/python/paddle/fluid/trainer.py#L115-L117

But the init parameter optimizer is an object, it will be created outside the with scope, this is not right, so we should make optimizer also a function call that return an optimizer boject

the interface should be

class Trainer(object):
    def __init__(self,
                 train_func,
                 optimizer_func,
                 param_path=None,
                 place=None,
                 parallel=False):