Created by: typhoonzero

Fix https://github.com/PaddlePaddle/Paddle/issues/6286

Multi trainers still have some problem, will refine in next PR.