关于recurrent_group step function输入输出的问题
Created by: rulai-huiyingl
直接使用 https://github.com/PaddlePaddle/models/tree/develop/nested_sequence/text_classification 双层序列文本分类的问题。 训练的时候查看一下层输出:
if isinstance(event, paddle.event.EndIteration):
output = event.gm.getLayerOutputs(['nest_group', 'word', 'emb', 'cnn_output'])
这几个名字对应的层如下: https://github.com/PaddlePaddle/models/blob/develop/nested_sequence/text_classification/network_conf.py :
data = paddle.layer.data(
"word", paddle.data_type.integer_value_sub_sequence(dict_dim))
emb = paddle.layer.embedding(name='emb', input=data, size=conf.emb_size)
linear_proj = paddle.layer.fc(name='cnn_output',
input=[conv3, conv4],
size=hidden_size,
param_attr=[fc_param_attr, fc_param_attr],
bias_attr=fc_bias_attr,
act=paddle.activation.Linear())
发现output['cnn_output']['value'].shape=(numberof_sentence_in_paragraph, hidden_size)
yield 的数据:
[[5130, 44, 72, 7, 1098, 14, 0, 887, 131, 153, 5130, 18, 283, 753, 0, 5130, 3, 5130, 2150], [37, 5130, 45, 5, 72, 4045, 377, 158], [45, 5, 32, 1300, 303, 35, 206, 2149, 564, 14, 2, 252, 26, 148, 61, 374, 1, 632, 5130, 5, 30, 0, 50, 3293, 7, 2160, 14, 0, 5130], [6, 6, 17, 457, 24, 4, 128, 11, 0, 887, 131, 44, 2, 50, 5130, 733], [5130, 5, 2, 222, 3707, 7, 10, 1167], [378, 10, 5, 5130, 8, 5, 1345, 13, 2, 5130, 1, 249, 8, 97, 4212, 56, 2311, 0, 912], [6, 6, 10, 16, 5, 5130, 5130, 7, 50, 71, 27, 276, 27, 108, 1054, 0, 2611, 4086], [3178, 17, 1688, 1, 2556]]
训练时候batch_size=1抓的输出
>>> output['cnn_output']['value'].shape
(8, 128)
按照教程说的recurrent_group会把输入序列打散以后传给step function,那么这里的cnn_cov_group函数的输入应该是单个句子中词的embedding(dense_vector_sequence)吧?输出应该是单个句子的向量,也就是(1,128)。为什么是所有句子的向量((numberof_sentence_in_paragraph, hidden_size))呢?
谢谢!