Created by: typhoonzero

Fix https://github.com/PaddlePaddle/Paddle/issues/4062

This is WIP, will add GPU code in next PR.