Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 2 年 前同步成功

通知 2325
Star 20933
Fork 5424
P
Paddle

optimize lod_reset op to avoid data transform !21271

!21271 已合并 由 saxon_zh@saxon_zh 创建

Created by: zhangting2020

问题来源(issue)： PaddlePaddle/benchmark#205 (comment) 问题描述(description)： lod_reset的Kernel实现中，不会访问输入X的数据；输入Y如果在GPU上就必须copy到CPU上访问。 (For lod_reset kernel, the data of Input X will not be used and the Input Y must be copied to the CPU if GPU mode is set.) https://github.com/PaddlePaddle/Paddle/blob/6fc3e8ec84af9e9fa2f191cbcc402de080962a67/paddle/fluid/operators/lod_reset_op.h#L48-L52

由于使用了默认的GetKernelTypeForVar，当输入X在CPU上，设置GPU执行，X发生CPU->GPU的数据拷贝，是不必要的；对于Y，如果在CPU上，会发生CPU->GPU的data transform，之后在kernel的执行过程中，又发生GPU->CPU的拷贝。所以X和Y不应该做data transform。 (Since the default GetKernelTypeForVar is used, if the input X is in the CPU and GPU mode is set, then data transform from CPU to GPU for X is unnecessary. If Y is in the CPU, the data transform from CPU to GPU will occur. During the execution of the kernel, the data will be copied from GPU to CPU. So the data transform for X and Y should be avioded.)

优化方案：通过重写GetKernelTypeForVar，返回的OpKernelType和期望的OpKernelType的place一致，避免在data transform过程中对X或Y进行CPU->GPU的数据拷贝。

测试代码(test code) 以X为例（take X as an example）

import paddle.fluid as fluid
import numpy

x = fluid.layers.fill_constant(shape=[6], value=2.0, dtype='float32', force_cpu=True)
y = fluid.layers.data(name='y', shape=[6], lod_level=2)
output = fluid.layers.lod_reset(x=x, y=y)

place = fluid.CUDAPlace(0)
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

y_ndarray = numpy.ones([6]).astype(numpy.float32)
y_lod = [[2, 2], [2, 2, 1, 1]]
y_tensor = fluid.create_lod_tensor(y_ndarray, y_lod, place)

res, = exe.run(fluid.default_main_program(),
               feed={'y':y_tensor},
               fetch_list=[output],
               return_numpy=False)
print(res)

修改前(before)：

I1120 14:49:20.509287 24082 operator.cc:172] CUDAPlace(0) Op(fill_constant), inputs:{ShapeTensor[], ShapeTensorList[]}, outputs:{Out[fill_constant_0.tmp_0:float[6]({})]}.
I1120 14:49:20.509335 24082 operator.cc:986] expected_kernel_key:data_type[float]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[PLAIN]
I1120 14:49:20.509364 24082 operator.cc:1081] Transform Variable fill_constant_0.tmp_0 from data_type[float]:data_layout[NCHW]:place[CPUPlace]:library_type[PLAIN] to data_type[float]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[PLAIN]
I1120 14:49:20.509384 24082 scope.cc:164] Create variable fill_constant_0.tmp_0
I1120 14:49:20.509404 24082 data_device_transform.cc:21] DeviceTransform in, src_place CPUPlace dst_place: CUDAPlace(0)
I1120 14:49:20.509460 24082 tensor_util.cu:122] TensorCopySync 6 from CPUPlace to CUDAPlace(0)
I1120 14:49:20.509703 24082 operator.cc:172] CUDAPlace(0) Op(lod_reset), inputs:{X[fill_constant_0.tmp_0:float[6]({})], Y[y:float[6]({{0, 2, 4}{0, 2, 4, 5, 6}})]}, outputs:{Out[lod_reset_0.tmp_0:float[6]({{0, 2, 4}{0, 2, 4, 5, 6}})]}.

修改后(after)：

I1120 16:07:01.309562 14423 operator.cc:172] CUDAPlace(0) Op(fill_constant), inputs:{ShapeTensor[], ShapeTensorList[]}, outputs:{Out[fill_constant_0.tmp_0:float[6]({})]}.
I1120 16:07:01.309648 14423 operator.cc:986] expected_kernel_key:data_type[float]:data_layout[ANY_LAYOUT]:place[CUDAPlace(0)]:library_type[PLAIN]
I1120 16:07:01.309715 14423 operator.cc:172] CUDAPlace(0) Op(lod_reset), inputs:{X[fill_constant_0.tmp_0:float[6]({})], Y[y:float[6]({{0, 2, 4}{0, 2, 4, 5, 6}})]}, outputs:{Out[lod_reset_0.tmp_0:float[6]({{0, 2, 4}{0, 2, 4, 5, 6}})]}.