训练中reshape op出现shape不对应
Created by: XipengY
- faster rcnn 架构 修改的shuffle net backbone
- 平台:centros6.3
- paddle版本号:1.5.0
- GPU:K40
- python2.7
训练信息
1)单机/多机
reshape的时候,维度对不上: 注:训练了大概10多个iter报错的,不是一开始就报错了。
错误信息如下:
Python Callstacks:
File "/lib/python2.7/site-packages/paddle/fluid/framework.py", line 1748, in append_op
attrs=kwargs.get("attrs", None))
File "lib/python2.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "lib/python2.7/site-packages/paddle/fluid/layers/nn.py", line 6797, in reshape
"XShape": x_shape})
File "Paddle_models/PaddleCV/PaddleDetection/ppdet/modeling/backbones/snet.py", line 270, in channel_shuffle
x = fluid.layers.reshape(x=x, shape=[batchsize, groups, channels_per_group, height, width])
File "PaddleCV/PaddleDetection/ppdet/modeling/backbones/snet.py", line 380, in inverted_residual_unit
return self.channel_shuffle(out, 2)
File "PaddleCV/PaddleDetection/ppdet/modeling/backbones/snet.py", line 106, in __call__
benchmodel=2,name=str(idxstage+2)+'_'+str(i+1))
File "PaddleCV/PaddleDetection/ppdet/modeling/architectures/faster_rcnn_thunder.py", line 70, in build
body_feats = self.backbone(im)
File "PaddleCV/PaddleDetection/ppdet/modeling/architectures/faster_rcnn_thunder.py", line 125, in train
return self.build(feed_vars, 'train')
File "tools/train.py", line 102, in main
train_fetches = model.train(feed_vars)
File "tools/train.py", line 234, in <module>
main()
C++ Callstacks:
Enforce failed. Expected output_shape[unk_dim_idx] * capacity == -in_size, but received output_shape[unk_dim_idx] * capacity:-384000 != -in_size:-432000.
Invalid shape is given. at [/paddle/paddle/fluid/operators/reshape_op.cc:131]
组网中打印的维度和预想的一致:
name: "concat_0.tmp_0"
type {
type: LOD_TENSOR
lod_tensor {
tensor {
data_type: FP32
dims: -1
dims: 60
dims: 40
dims: 40
}
lod_level: 0
}
}
但是训练中打印了tensor,确实有奇怪的现象:
Tensor[concat_0.tmp_0]
shape: [6,60,33,40,]
dtype: f